If you tap a single neuron inside a language model and watch when it fires, you usually get a mess. The same neuron might light up for French text, for DNA sequences, and for HTTP headers, with no obvious thread connecting them. That is awkward for anyone hoping to read a model's internals, because the natural unit we can point at, the neuron, doesn't line up with the natural unit of meaning, the concept. Sparse autoencoders (SAEs) are the tool built to pull those tangled neurons apart into something closer to one concept at a time. Where a linear probe reads information out of a layer and activation patching intervenes on it, an SAE tries to decompose it, and it has quickly become one of the most talked-about tools in the field (Bricken et al., 2023; Cunningham et al., 2024).
The problem: superposition
A neuron that responds to several unrelated things is called polysemantic, and it turns out to be the rule rather than the exception. The leading explanation, laid out in Anthropic's toy models of superposition, is that models represent more distinct features than they have neurons. If you have a hundred neurons but a few thousand features worth tracking, you can't hand each feature its own private neuron. Instead the features share, encoded as directions in activation space that are close to, but not exactly, orthogonal. Any single neuron then sits at the crossing point of several of these directions, which is exactly why it appears to answer to unrelated inputs.
The idea: an overcomplete dictionary
The fix an SAE proposes is to stop reading neurons and start reading directions. Take the activation vector at some layer and re-express it as a combination of entries drawn from a large learned dictionary, under one firm rule: only a handful of entries may be active at once. Concretely, an encoder maps the activation into a much wider vector, most of whose entries are forced to zero, and a decoder adds the few active entries back up to reconstruct the original activation. The whole thing is trained to keep the reconstruction close to the input, ‖x − x̂‖², while a sparsity penalty pushes almost everything to zero.
Those dictionary entries are the features, and this is where the payoff is meant to come from. The dictionary is deliberately much wider than the activation it describes, often eight to thirty times as wide, which gives the model room to give each concept its own entry instead of forcing them to share a neuron. The sparsity rule is what makes the result readable: at any given token only a few features are on, so instead of a dense block of numbers you get a short list of switched-on things (Cunningham et al., 2024). The hope, and it is still partly a hope, is that those things turn out to be monosemantic, each firing for one coherent concept.
A clean example: the Golden Gate feature
The most vivid demonstration came from Anthropic's Scaling Monosemanticity work on Claude 3 Sonnet. Among millions of learned features was one that fired on mentions of the Golden Gate Bridge, in text, in code, and across languages. The convincing part wasn't just that it fired on the right inputs. When the researchers clamped that single feature to a high value, the model started steering nearly every conversation toward the bridge, at one point insisting it was the Golden Gate Bridge. You can browse this kind of feature-level catalogue yourself on Neuronpedia, which hosts auto-generated descriptions for the features of many open SAEs.
This is the same move that made the Othello board probe convincing: a direction earns its interpretation not only by correlating with a concept, but by changing the model's behaviour when you turn it up or down. An SAE gives you a whole dictionary of such directions at once, which is what sets it apart from probing for one property at a time.
ReLU versus JumpReLU
Where does the sparsity actually come from? From a threshold on the encoder's output. The original SAEs use a plain ReLU, which zeroes out every negative pre-activation and passes the rest through unchanged. A newer variant, JumpReLU (Rajamanoharan et al., 2024, used in DeepMind's Gemma Scope release), adds a hard cutoff: anything below a learned threshold θ is set to zero, so weak, barely-positive activations no longer count as a feature firing at all.
This sounds like a minor engineering choice, and for reconstruction quality it mostly is. But that threshold is also a margin against noise, and how much margin an SAE has changes how it behaves when the activations it reads are perturbed, something I'll come back to in a follow-up post.
The catch
As with every tool in this series, it's worth being clear about what an SAE does and doesn't give you. A good reconstruction is not the same as good understanding: an SAE can rebuild the activation almost perfectly while some of its features remain uninterpretable, or fire for reasons that resist any clean description. Reconstruction measures whether the dictionary can represent the activation, not whether the entries mean anything.
There is also the question of granularity. Grow the dictionary and concepts tend to split: a single "legal language" feature in a small SAE fractures into a dozen narrower ones in a larger SAE, and there is no obvious right answer for how fine is fine enough (Bricken et al., 2023). Train the same SAE twice with different random seeds and you don't get the same dictionary either, so the features you name are partly an artifact of one training run. And most of these results come from full-precision models, whereas the models actually deployed are usually quantized to save memory, which raises a question the interpretability literature is only starting to ask: do the features survive when you compress the model underneath them? That last gap is where my own work lives, and it's the subject of the next post.
Where this leaves us
Sparse autoencoders are the decomposition tool of the trio: probing reads a property out of a layer, patching intervenes to test whether the model uses it, and an SAE tries to break the whole activation into a vocabulary of concepts you can then read and steer. It's the most promising route anyone has found past the polysemantic-neuron problem, and the fact that you can clamp a feature and watch the model's behaviour bend to it is genuinely striking. But it is a young tool, leaning on assumptions, monosemanticity, the right dictionary size, stable features, that are still being tested. Reading a model through an SAE means reading it through a dictionary that was itself learned, with all the room for error that implies. If you want the sharper version of that worry, about what happens to these features once the model leaves the lab, that's where I'm headed next.