Finding GPT-2’s induction heads is easy. The usual detector is a bad guide to which one matters.

Every other note in this series treats the model as a black box: prompt in, behavior out, which is all the commercial models let you see. This one opens a model up. GPT-2-small is small enough and open enough to run on a laptop with the weights exposed, so you can watch individual attention heads work, turn them off one at a time, and see what breaks. I wanted to start with the most famous result in mechanistic interpretability and check whether the standard tool for it does what people use it for.

That result is induction heads. If you show a transformer a sequence and then repeat it, certain attention heads notice the repeat and copy it forward: seeing “…the cat sat on the mat … the cat sat on the”, they look back to what followed “the” last time and predict “mat”. They are a big part of how models do in-context learning. The standard way to find them is the induction score: feed a repeated random sequence and measure, for each head, how strongly it attends back to the token that followed the current one last time. High score, induction head. It is a clean, cheap, attention-pattern detector, and it is everywhere in the field.

The question I pre-registered: the score is an attention pattern, and attention is not the same thing as causal importance. Does the score that names the induction heads actually tell you which head matters?

The score works: it finds the textbook heads

First the replication, and it is clean. Ranking all 144 heads of GPT-2-small (12 layers, 12 heads each) by induction score on repeated random sequences, the top of the list is exactly the cast the literature names: heads 5.5, 6.9, 5.1, 7.10, 7.2, all scoring at or above 0.83 out of 1, all in the middle layers where induction is supposed to live. The detector does its job. If all you want is “which heads have the induction attention pattern,” the score answers it.

Then I turned each head off, and the two rankings barely agree

The causal test is ablation: replace one head’s output with its average and measure how much worse the model gets at the induction task (its loss on the repeated half, which a working induction circuit makes almost free). Rank all 144 heads by that damage and you have a second list, “which head does the model actually need.” If the score means what people use it to mean, the two lists should look alike.

They do not. The rank correlation between induction score and causal effect across the 144 heads is 0.14. Of the top ten heads by score, exactly one is also in the top ten by causal effect. I had pre-registered that the two would agree substantially, a correlation of at least 0.4, on the reasoning that the score is a decent proxy. That was wrong, and wrong in the direction of the score being a worse guide than I expected. The detector finds the induction heads; it is close to useless for ranking them.

Two things drive the gap, and both are checkable.

Reason one: the induction heads back each other up

Take head 7.10. It has the fourth-highest induction score in the model, a textbook induction head by the attention pattern. Turn it off by itself and the induction loss barely moves: it ranks 113th of 144 by causal effect. Head 10.7, tenth by score, ranks dead last, 144th, by effect. Turning these “induction heads” off one at a time does almost nothing.

But turn off the top ten by score all at once and induction collapses: the loss on the repeated half jumps from 0.52 to 8.19, from near-free to badly broken, about sixteen times worse. So the score is not finding fake heads. It is finding a redundant set. Several heads do the same job, so removing any one is covered by the others, and single-head ablation, the obvious causal test, systematically under-rates every one of them. The attention pattern sees the redundancy; the one-head-at-a-time knockout cannot.

Reason two: the head that matters most is invisible to the score, and I cannot cleanly say what it does

Now the other direction. The single most causally important head for induction in GPT-2-small is head 0.3, in the very first layer. Turning it off nearly triples the induction loss, the largest effect of any head in the model. Its induction score ranks it 134th of 144, near the bottom.

It scores low because it does not have the induction attention pattern, and the score only looks for that pattern. I first assumed 0.3 was the textbook previous-token head, the standard layer-0 input to an induction circuit, the head that points each position back at the one before it. So I checked its attention, and it is not: on ordinary text 0.3 attends mostly to itself, about four fifths of its weight, and only about one part in ten to the previous token. My tidy mechanistic story was wrong. What I can say for certain is causal and specific: turning 0.3 off raises the induction loss by 1.20 while changing the model’s loss on ordinary text by -0.03, essentially nothing, so it breaks induction in particular and not the model in general. Every high-effect head I ran that control on came back the same way, induction-specific rather than general disruption, and they span the early and middle layers, not just layer 0. So the score misses real, induction-specific contributors at several depths, and the most important one is a head whose actual mechanism I would have to do more work to name. Ablation found the head; it did not hand me the story, it took away the story I had assumed.

What this does and does not show

The honest scope first. This is one small model and one circuit, and ablation is one intervention among several (mean-ablation can mislead in its own ways, and I leaned on the natural-text control exactly because of that). I am not claiming the induction score is useless; it is a fast, cheap way to locate the heads that carry the induction pattern, and it found them correctly. The claim is narrower and, I think, more useful: the attention-pattern score finds components, but it does not rank their causal importance, and reading it as if it does gets you a list that correlates 0.14 with what the model actually needs. It over-credits a redundant cluster that no single member is required for, and it completely misses the first-layer head that matters most.

This is the white-box version of a thing the black-box notes kept finding too: the obvious measurement and the real one come apart, and you only know which is which when you intervene. There the gap was between what a model says and what it does. Here it is between what a head looks like it is doing and what the model falls apart without. The lesson rhymes. If you want to know what a part is for, you have to take it out, and even then you have to take out the right combinations and check that the damage is specific.

Method, for anyone who wants to check it. GPT-2-small through TransformerLens, run locally on a laptop, $0, no API. Induction score and per-head mean-ablation effect computed over the same seeded repeated-random batch; the score-versus-effect rank correlation, the joint ablation of the top heads, and the natural-text specificity control are all deterministic and reproduce from the saved script. The pre-registration (with the correlation I predicted and got wrong), the code, and the full 144-head table are in the experiment directory.

This is the first white-box note in a series on how AI models behave when you measure them honestly. The others, and the through-line, are on the trutina landing page.