8 entries tagged "gpu" — 2 posts, 6 links.
This is the runtime stack link for the gpt-oss moment: kernels from the Hub, MXFP4, tensor and expert parallelism, dynamic KV cache behavior, continuous batching, and faster loading.
Worth keeping because it connects model release excitement to the boring but decisive parts of deployment: memory, cache shape, batching, and what hardware the trick actually runs on.
This is worth keeping because local and Windows-side inference are no longer edge cases for product teams. Hardware choices now shape who can prototype, test, and deploy models without waiting for a cloud GPU lane.
The guide is beginner-friendly, so it is not the deepest systems reference. Its value is showing where AMD's PyTorch story is becoming practical enough to track.
This belongs next to any local inference or training work because cross-platform ML is usually where clean notebooks meet real machines.
The useful lesson is not only PyTorch-specific. Runtime packaging, hardware backends, and dependency friction are product risks when your users are expected to run models outside your exact environment.
This is a strong local-inference link because it shows the gap between framework support and hardware-specific performance work. Apple Silicon can be serious, but the path often goes through kernels, not just model.to("mps").
The AI-generated-kernel angle is interesting, but the bigger point is operational: when the backend is the bottleneck, model.to("mps") is only the start.
This is reference-quality material for anyone trying to reason about accelerator performance without hand-waving. It explains GPUs as machines with memory, bandwidth, communication, and parallelism constraints.
Useful far beyond JAX. The same mental model helps with PyTorch, Transformers, NeMo, and any conversation where "just add GPUs" is hiding the actual bottleneck.
This belongs in the queue because fine-tuning and inference decisions increasingly depend on numerical formats, not only model architecture. NVFP4 is a reminder that performance work moves through the whole stack: math representation, kernels, hardware, training stability, and serving economics.
The tradeoff is hardware specificity. A format can be a breakthrough and still be irrelevant to a Mac-local or non-NVIDIA deployment path.