Tag: machine learning

Edward Yang offers a clarifying model for large-scale ML work: keep approachable PyTorch code as an executable reference, while specialized kernels and explicit forward-backward paths become the production implementation. That separates research ergonomics from hardware-specific optimization without discarding the former. The reference is not dead weight; it is the thing against which faster code can be checked.

The Hacker News objection that this sounds like giving up on compilers is fair, but it frames the trade too narrowly. A compiler remains valuable where it can reliably optimize and preserve intent. The harder cases need explicit control over data movement and fusion, plus a verifier stronger than "the loss looked plausible." Treating the reference implementation as a spec makes that specialization accountable rather than merely faster.

Serge Zaitsev builds a compact tensor library in C from the representation upward: contiguous data, shapes and strides, broadcasting, kernels, SIMD, then enough layers and manual backward passes to train small models. It is not a PyTorch replacement, which is exactly why it works so well as a teaching artifact. Once the runtime is small enough to read, "tensor operations" stop feeling like API magic and become memory layout, indexing, and carefully composed loops.

The Hacker News discussion catches an important boundary: a tensor in a numerical library is a useful computational representation, but that is not the full mathematical object, with its coordinate-independent transformation behavior. That is not a reason to reject the simplification. It is a reason to label it clearly. Good systems explanations earn their clarity by saying which layer they are describing, and this post is strongest when read as a tour of the engineering layer beneath modern ML tools.

This is a beginner resource, but it is worth keeping because feature, label, target, and ground truth confusion still causes real project mistakes.

Useful as a glossary-level companion to the ML terminology posts. The caveat is that the definitions are introductory, so they should support judgment rather than replace model-review rigor.

PCA is one of those concepts that gets overused and under-explained. This is a useful reference for the intuition, terminology, and tradeoffs around dimensionality reduction.

Keeping it because PCA sits at a nice intersection for this site: linear algebra, exploratory data analysis, feature review, and the danger of compressing structure before you know what matters.