What changed since July

In July I published a set of experiments on how AI models review code, grade each other and judge their own confidence. Models have moved on since then, so I ran the three core experiments again on four current Claude models: Haiku 5.5, Sonnet 5.5, Opus 5.5 and Fable 5.2. The rules for each rerun were written down and committed before any model was called.

The short version: one finding held, one got a sharper limit, and one test is now too easy to tell these models apart.

1. A second look beats a different model, inside one family

In June three reviewers from three different companies, working blind, together caught all 30 planted bugs in 20 code snippets. The best single reviewer caught 27.

The question for the rerun came from outside. A study published in October (Song, 2026) found that a second model was not clearly better than the same model looking again in a fresh session. So I tested exactly that: does pairing two different Claude models catch more than asking the same model twice?

Now. Each of the four models reviewed all 20 snippets twice, in separate fresh sessions.

Pairing Bugs caught by the pair Extra over the better of the two
Same model, two looks 26.00 of 30 0.50
Two different models 26.25 of 30 0.54

The difference is 0.04 bugs, with a 95% range of -0.58 to 0.67. By the rule I set in advance, that reads: a second look is just as good. Taking one review from each of the four models, the group caught 26 to 29 depending on which reviews were picked. One bug was missed by every review, eight times out of eight.

Single models ranged from 23 to 27 of 30. The smallest model was not the weakest reviewer.

What this does not say: it does not test reviewers from different companies. The June result still stands as measured, and I did not rerun it. What it adds is a limit. Within one model family, swapping tiers buys you nothing a fresh session would not.

2. Graders still lean toward their own writing

In July Claude Opus 4.8, grading blind in a panel of three model families, gave its own answers +2.7 points more than the same answers earned from others.

Now. Four Claude models each wrote answers to ten prompts, then each graded all of them blind, with the labels shuffled.

Grader Extra points for its own answer 95% range
Haiku 5.5 +3.65 +1.63 to +6.07
Sonnet 5.5 -1.05 -2.67 to +0.48
Opus 5.5 -0.50 -1.63 to +0.68
Fable 5.2 +1.90 +0.52 to +3.40
All four, averaged +1.00 +0.32 to +1.77

Points are on a 100-point scale, holding the answer fixed and removing how generous each grader is overall. The average is above zero, so by the rule set in advance: Claude models favour their own model’s text, even against close relatives. Two of the four do it clearly. Two show nothing detectable at this size, which is not proof of none.

The effect is small, about one point in a hundred. It is still a reason to keep the first lab rule. Larger outside studies agree that the lean is real and that a panel of judges shrinks it without removing it (Pombal and others, 2026).

The July and November numbers come from different panels, so only the direction can be compared. The direction is the same.

3. The confidence test is now too easy

In July Opus 4.8 answered 22 hard questions, got 95.5% right and said it was 99% sure on average. Slightly overconfident.

Now. The same 22 questions plus a harder bank of 72. The table shows the harder bank; on the original 22 every model scored 100%.

Model Right, harder bank Average confidence
Haiku 5.5 98.6% 90.5%
Sonnet 5.5 98.6% 95.1%
Opus 5.5 98.6% 94.6%
Fable 5.2 100% 96.0%

Every model scored above 90%, which the plan had already named as a ceiling result: these questions no longer separate these models. All four now state slightly less confidence than their scores would justify, the opposite of July. I would not lean on that. When nearly every answer is right, any confidence short of 100% looks like modesty.

Two honest limits. The answers to these questions have been public since July, so newer models may have seen them. And with only one or two wrong answers per model, there is nothing to say about whether models share the same mistakes. That question matters: an earlier study found that when two models are both wrong they give the same wrong answer about 60% of the time (Kim and others, 2025). Testing it on current models needs harder, private questions.

Where each July claim stands

Claim Status Why
A model should never grade its own work Holds The lean toward own text showed up again
Different reviewers cover each other’s blind spots Holds, with a limit True across companies in June. Within Claude, a second look does as well
Top models are overconfident on hard questions Not measurable here The question bank hit its ceiling
GPT-5.5 favours its own work most Not retested Only Claude models were run this round
Everything else on this site Not retested Dated July 2026 and should be read that way

What I would do with this

How it was run

The plans, raw answers, session checks and results are in the repositories, under round3/ in decorrelation-study and experiments/claude-tiers-2026-11/ in trutina.