More features than dimensions

Open up an MLP layer in a language model and look at what makes a single neuron fire. You will often find inputs that have nothing to do with one another: academic citations, Korean text, HTTP requests.This part follows the review Sparse autoencoders for language-model interpretability, bitfilter 0.29, 12 November 2026. Primary sources are cited in the margin as they come up. A neuron like this is called polysemantic, and it is a problem for anyone who wants to explain a model in terms of its parts. A part that means three unrelated things cannot be named, counted or switched off cleanly.

If neurons are the wrong units, what are the right ones? The working answer since 2022 is that the meaningful variables are directions in activation space, and that there are many more of them than there are neurons.

Superposition

A layer of width $n$ has room for only $n$ orthogonal directions. But it can hold many more directions that are almost orthogonal, at the cost of some interference between them. If each feature is active only rarely, two features seldom fire together, and the interference seldom matters. Under those conditions it pays to store more features than there are dimensions.Elhage et al., Toy Models of Superposition, 2022. In their small ReLU models, past a sparsity threshold the network switches sharply from storing one feature per dimension to storing many, arranged in regular geometric figures such as the pentagon below.

The figure makes this concrete with fixed weights. There are 5 features, written into two dimensions along evenly spaced directions $d_i$ and read back as $\hat x_i = \mathrm{ReLU}(d_i\cdot h - t)$. Compare this with the honest alternative: store two features exactly and drop the rest.

When each feature is active 8% of the time, packing wins clearly. Raise that chance far enough and the curves cross: once several features are often active together, interference costs more than dropping features outright. With five features and the default threshold the crossing comes at about one in two; with three features, at about one in six. The threshold $t$ is what makes packing work at all. A negative bias filters out the small, spurious readings that interference produces.

If real models behave like this, an activation $x \in \mathbb{R}^n$ is approximately a sparse combination of many directions:

$$x \;\approx\; \sum_{i=1}^{m} f_i\, d_i, \qquad m \gg n, \qquad \lVert f\rVert_0 \ll m .$$

Finding the $d_i$ and the $f_i$ from activations alone is a sparse dictionary-learning problem.

Back to dictionary learning

Piece 4 ended with exactly this problem: an overcomplete dictionary and a sparse code, learned from data, with an encoder that produces the code in one forward pass. Dictionary learning has been applied to word embeddings before, where sparse combinations of “discourse atoms” track word senses,Arora et al., 2016 (TACL 2018). and to transformer activations, as “transformer factors”.Yun, Chen, Olshausen and LeCun, 2021. The direct precursor of the SAEs in this part was a one-layer autoencoder with an L1 penalty on its hidden layer, which recovered the ground-truth features of a toy model like the one above.Sharkey, Braun and Millidge, interim report on the AI Alignment Forum, December 2022. Within a year the method was being applied to real language models.