What’s established (the foundation)
Our approach builds directly on two Geva et al. papers:
Geva et al. 2021 — “Transformer Feed-Forward Layers Are Key-Value Memories”: Established that each column of down_proj (what they call a “value vector”) induces a distribution over the vocabulary when projected through lm_head. They showed these value vectors correspond to interpretable token clusters. This is where our dot[t] = lm_head[t] · value_i comes from — it’s their core operation.
Geva et al. 2022 — “Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space”: The direct predecessor to what we do. They decompose each FFN layer’s output into per-neuron sub-updates in vocabulary space, and show each sub-update promotes an interpretable concept (e.g. {Paris, France, French, European}). They view the residual stream as a changing distribution over vocabulary, with each cell adding or subtracting from specific tokens’ logits. This is the methodological basis for our per-cell, per-verb logit decomposition.
Dai et al. 2022 — “Knowledge Neurons in Pretrained Transformers”: Introduced “knowledge neurons” — specific MLP hidden units that express factual knowledge — and a knowledge attribution method to identify them.
Subject-verb agreement circuits across languages (EMNLP 2024): Directly examines per-neuron MLP contributions to verb agreement, including the sign of neuron activation (positive/negative) determining whether it promotes plural vs singular verb forms. Very close to our setup but for number agreement rather than tense.
What we added (what’s novel in our table)
The established technique is: project value vectors through lm_head, get token distributions. What our directionality table does differently:
1. Conditioning on tense via the signed mean activation. Geva et al. project value_i through lm_head and report the top tokens — that shows what the cell could write. We multiply by mean_past[i] and mean_present[i] separately, showing what the cell actually contributes on each tense class. This is where the demotion becomes visible: a cell whose value points at was but fires negatively on present inputs demotes was on present — something invisible in the unsigned Geva projection.
2. The differential Δpast − Δpresent as a per-token tense attribution. Geva et al. don’t contrast two input classes. We compute (mean_past - mean_present) × dot[t] for every verb token, giving a signed, per-verb, per-cell tense attribution. This is what surfaces the three mechanisms (direct promotion, competitor suppression, intra-tense competition) and lets us classify cells by their functional role.
3. Scanning the full verb lexicon instead of reporting top-k tokens. Geva et al. report the top-k most-promoted tokens per value vector (which are usually nouns, function words, etc. — verbs are rarely rank-1). We pre-encode the entire verb lexicon and compute lm_head · value for all verb tokens, thresholding on |Δpast − Δpresent|. This is why we see full morphological clusters (126 present verbs in one cell) that a top-k approach would miss entirely.
Summary
| Aspect | Prior work | Us |
|---|---|---|
lm_head · value_i → token distribution | Geva 2021, 2022 | same |
| Per-neuron sub-updates to vocabulary | Geva 2022 | same |
| Signed activation × value = actual contribution | Implicit in Geva 2022 | explicit, conditioned on past vs present input class |
Δpast − Δpresent per verb per cell | not done | novel — gives signed tense attribution at the (cell, verb) level |
| Full verb-lexicon scan with threshold | not done (top-k tokens) | novel — surfaces complete morphological clusters |
| Cross-model comparison of cell mechanism types | not done for tense | novel |
So the foundation (value-vector projection, per-neuron vocabulary decomposition) is Geva et al. 2021/2022. What’s new is conditioning on tense class to get signed directionality, the per-verb differential attribution, and the full-lexicon threshold scan that reveals the complete verb clusters per cell. The “present = not-past by suppression” finding is an empirical result that comes from this methodological extension — it’s invisible in the original Geva-style unsigned projection.
Sources:
- Geva et al. 2021 — Transformer Feed-Forward Layers Are Key-Value Memories (EMNLP 2021)
- Geva et al. 2022 — Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space (EMNLP 2022)
- Dai et al. 2022 — Knowledge Neurons in Pretrained Transformers (ACL 2022)
- Subject-verb agreement circuits across languages (EMNLP 2024 Findings)
- Logit Prisms: Decomposing Transformer Outputs for Mechanistic Interpretability
- MLPs in Transformers — Learn Mechanistic Interpretability