About
Out of Sample is a course in applied neural networks, for students at master’s level or starting a PhD.
The course is about what happens when you actually train a network: why a step size works or diverges, why a model that fits its training data can still fail on new data, how to adapt a large pretrained model without retraining it. Each piece takes one such question, works through the mathematics that answers it, and puts the answer in a figure you can change.
The figures are the point. Most of what goes right or wrong in training can be seen in two or three dimensions, and a dial you can push teaches the threshold faster than the derivation alone. The derivation is still there, and the exercises at the end of each piece ask you to use both.
It assumes linear algebra, multivariable calculus and basic probability, and enough Python to follow a short PyTorch snippet. Each piece says on its title page what it builds on.
The parts
Written by
Aleksandar Bradic
Downhill, The part you didn’t fit, A small change to a large matrix, What survives the bottleneck, Compress, then reconstruct, Which axes?, A distribution over codes, Wider than the input, More features than dimensions, A wide, sparse dictionary, Training one, How latents switch on, Thirty thousand times larger, From features to circuits, Measuring a dictionary, Ways a dictionary misleads, What SAEs are good for