A growing line of mechanistic interpretability work has shown that language models sometimes represent simple counting variables, like how many characters into a word you are, on a curved, roughly one-dimensional manifold, with individual attention heads doing the geometric work needed to move along that curve. It's a nice result, but a narrow one: one task, on one kind of count. The question we wanted to answer is how general this is. Is "represent it as a curve" something models do for ordinal information in general, or is character-counting a special case?
The setup
We picked four tasks that all share the same basic shape (each one asks the model to track where something is along an ordered axis) but that differ in how that position has to be computed:
- Bracket depth: how deeply nested the current token is inside brackets.
- Indentation: how many levels of indentation a line of code sits at.
- Table position: which row or column a token belongs to.
- Numeric magnitude: the relative size of a number mentioned in text.
We ran these across Gemma-2-2B, Gemma-2-9B, and Qwen3-4B, and looked at the geometry of the resulting representations using probing and dimensionality analysis, then checked which parts of that geometry actually mattered using activation patching.
What we found
The dividing line that emerged wasn't about the four task names so much as about how the ordinal variable can be computed. For tasks where the value is essentially readable off the current token and a short local context (bracket depth, indentation), we consistently found tidy 1D curved manifolds, tiled by features that each respond to a narrow window along the ordinal axis. It's a "place-cell"-like code: think of each feature as a little bump function,
$$ f_i(x) \approx g(x - c_i) $$centered at its own location $c_i$ along the curve, so that the population of features together acts like a soft histogram of position. (This equation is illustrative, not a quoted result from the paper. Swap in your actual figure here.)
Tasks that instead require integrating information across multiple positions, or pulling an ordinal value out of something semantic (e.g. reading "twelve" and knowing it's bigger than "three"), didn't get this treatment nearly as cleanly. Their representations were higher-dimensional and far less manifold-like: less of a clean curve, more of a tangle.
Why architecture matters
The amount of geometric "twisting" models do to build these curves also varied a lot by architecture. Qwen3-4B showed 4–10× stronger twisting than the Gemma models on indentation. And within Qwen3-4B itself, that twisting wasn't applied uniformly: for indentation, the twisted representation preserved ordinal order well ($\rho \geq 0.77$), but for numeric magnitude it barely preserved order at all ($\rho \leq 0.23$). Same model, same general mechanism available, used selectively depending on the task.
Is it actually causal?
Geometry that just correlates with a variable isn't the same as geometry the model actually uses. To check causality, we patched activations within the identified manifold subspaces and confirmed that doing so changes model behavior in the direction the geometry predicts, across all four tasks. That's the piece that turns "we found a pretty shape" into "this shape is part of how the model computes the answer."
Open questions
A few things I'm still chewing on: whether "locally computable" is really the right dividing line or just a proxy for something else; how this connects to the broader manifold hypothesis in deep learning; and whether training dynamics push models toward these curved codes because they're efficient, or because they fall out of something else entirely. More on this soon.