What SAEs are good for

The pattern in the evidence is fairly consistent. When you know what you are looking for, a supervised method trained on that target usually does better. SAEs earn their keep when you don’t know what to look for.

Where they helped

Audits. In a blind audit of a model trained with a hidden reward-model-exploitation objective, three of four teams found it, and the successful teams used SAEs alongside other tools.Marks et al., Auditing Language Models for Hidden Objectives, Anthropic, 2025.

Emergent misalignment. A “toxic persona” feature predicts and controls the broad misalignment that appears after narrow fine-tuning. Fine-tuning on a few hundred benign samples restored alignment.Wang et al., Persona Features Control Emergent Misalignment, OpenAI, 2025.

Other kinds of model. On the protein model ESM-2, SAEs found up to 2,548 interpretable features per layer, linked to up to 143 biological concepts, against at most 46 neurons per layer linked to 15.Simon and Zou, InterPLM, 2024.

Cheap classifiers. A company reports raising PII-detection F1 from 51% to 96% with SAE probes, at 10 to 500 times lower cost than an LLM judge.Goodfire and Rakuten; this is the companies’ own claim.

Where they fell short

Probing and steering on known concepts (piece 11). Harmful-intent detection: dense probes were near-perfect, including out of distribution, and SAE probes transferred worse. Soon afterwards DeepMind’s interpretability team wrote that it was deprioritising fundamental SAE research.Smith et al., Google DeepMind, March 2025. The team also wrote: “we do not consider reducing approximation error to be a good proxy task.” Unlearning: removing knowledge by clamping several features had “similar or larger unwanted side-effects” than a fine-tuning method.Farrell, Lau and Conmy, 2024.

After dictionaries

Three lines of work move away from decomposing activations.Figures in this section are from the bitfilter review. Several are recent and not yet peer reviewed.

Parameter decomposition looks for sparse pieces of the weights rather than the activations. Goodfire decomposed a four-layer, 67M-parameter model trained on the Pile into 38,912 rank-one subcomponents, of which about 10,000 were alive and about 205 active per token.

Weight-sparse transformers are trained so that about one weight in a thousand is nonzero. OpenAI reports circuits roughly sixteen times smaller at the same loss. The aim is to build interpretability in during training rather than decode it afterwards.

Natural-language autoencoders replace the sparse code with a text description that must reconstruct the activation. On Claude models, Anthropic reports that descriptions explained 60–80% of variance after training, up from 30–40%.

Open questions

1. Does the model have its own units, or is every dictionary one of many equally valid bases?
2. What is in the dark matter: missing features, non-linear structure, or noise?
3. Can selecting latents by their output effect close the gap with supervised probing and steering?
4. Do SAE-based monitors survive adversarial inputs? Small input changes can alter the active concepts without notably changing the base model’s activations.
5. Does parameter decomposition scale to production models?

Where that leaves us

SAEs find human-readable features in every model they have been tried on, and they scale to tens of millions of latents. Architecture work has improved the sparsity–fidelity frontier, but those gains have not reliably carried over to downstream tasks. On known targets, simple baselines usually win. In open-ended audits and discovery, SAEs help find behaviour nobody knew to look for. Transcoders and attribution graphs explain part of the computation, and their error nodes show how much is left. As of November 2026, SAEs are a mature first-look tool, not a route to complete understanding of a model.