Out of Sample

Training a network is walking down a surface you cannot see, one gradient at a time. How big a step you take matters more than almost anything else.

A model judged on the data it learned from will always flatter itself. Hide some of the data, and the flattery stops.

Fine-tuning a pretrained model rarely needs to move every weight. LoRA trains a low-rank correction instead, and loses surprisingly little.

An autoencoder learns by copying its input through a code. Make the code narrow, make it a distribution, or make it sparse, and you get three different models that answer three different questions.

An autoencoder is trained to output its own input. That looks pointless until you constrain what happens in the middle. The simplest case, linear with a narrow code, turns out to be principal component analysis.

Reconstruction fixes the subspace an autoencoder uses but not the coordinates inside it. Any invertible change of basis reconstructs equally well. To read a code, something else has to pick the axes, and sparsity is one way to do it.

The variational autoencoder replaces a single code with a distribution over codes and a prior to match. That makes it a generative model. It also shows that the choice of prior is the choice of axes, and that an L1 penalty is a Laplace prior.

An overcomplete autoencoder can copy its input in many ways, so it needs another constraint. Sparsity is the oldest. A ReLU encoder turns out to be one step of a classical sparse-coding algorithm, unrolled and learned.

Neurons in language models mean several things at once. The superposition hypothesis explains why, and suggests that sparse dictionary learning can recover the meanings.

The sparse autoencoder in its original form, how it is trained, the three numbers every paper reports, and what the first experiments on language models found in 2023.

Watch a sparse autoencoder learn, see what the penalty, the width and the random seed do to it, and then the practical details that make training work on billions of activations.

Most of the architecture work since 2024 replaces the L1 penalty. Gated, TopK, JumpReLU and BatchTopK change the activation function; p-annealing, end-to-end training and Matryoshka change the objective.

Within eighteen months, dictionaries went from a thousand features on Pythia to tens of millions on frontier models. This piece covers the large runs, the open suites, and what scale costs.

An SAE describes what a layer represents. Transcoders, crosscoders and cross-layer transcoders try to describe what it computes, and attribution graphs chain them together to trace a single prompt.

Better reconstruction at a given sparsity has not reliably meant a more useful dictionary. This piece covers what benchmarks, automated explanations and comparisons with simple baselines show.

Absorption, hedging, composition, dark matter, dependence on the random seed, success on random networks, and features that are not linear. Together they suggest an SAE gives one useful basis among many.

Where sparse autoencoders have helped and where simple baselines beat them, what may replace activation dictionaries, and the questions still open in November 2026.