Tag: distributed-training

torch.compile, mixed precision, gradient accumulation, DDP vs FSDP, and the profiler — the five levers I reach for before rethinking the model architecture.

Outcome: Cut training wall-clock time and GPU memory pressure by applying compile, AMP, and accumulation patterns in sequence before ever touching model architecture.