Tag: training

3 entries tagged "training" — 2 posts, 1 link.

Posts

Oct 3, 2025 — 7 min — Platform & AI

The Three-Run Lab: How I Triage Slow PyTorch Training

A repeatable triage routine — the three-run baseline, DataLoader diagnosis, five profiler signatures, and a copy-paste scaffold — for finding where training time actually goes before touching the model.

pytorch ml

Sep 29, 2025 — 7 min — Platform & AI

PyTorch Training Throughput: The Patterns That Actually Move the Number

torch.compile, mixed precision, gradient accumulation, DDP vs FSDP, and the profiler — the five levers I reach for before rethinking the model architecture.

pytorch ml

Links

Articlejax-ml.github.ioApr 13, 2026Permalink

How to Think About GPUs

How To Scale Your Model

This is reference-quality material for anyone trying to reason about accelerator performance without hand-waving. It explains GPUs as machines with memory, bandwidth, communication, and parallelism constraints.

Useful far beyond JAX. The same mental model helps with PyTorch, Transformers, NeMo, and any conversation where "just add GPUs" is hiding the actual bottleneck.

All tags