Is Opus 5.5 nerfed? Compared with what?

Every few weeks someone announces that Anthropic has quietly nerfed its flagship model, and this month it's Opus 5.5's turn. A post claiming its reasoning changed on 3 November went viral, the GitHub issues are piling up, and Fable 5.2 arriving last week has only added fuel.

My view, which is not a fashionable one this week, is that almost nobody making the claim can test it. To say a model got worse you need something that measured it before and measures it the same way now. Compared with what, exactly?

What the fixed instruments say

Some people are doing this properly. LiveNerf runs the same 78 questions against Opus 5.5 every day through a pinned version of Claude Code, and has done since launch week. Fifty days in, it has found no change. AI Stupid Level reruns coding, reasoning and tool-use tests on a schedule and, as of this morning, reports no sustained decline.

The usual reply is that those are short, tidy test questions, while the complaints come from long agent sessions doing real work. That's fair, and it cuts both ways, because a long agent session is exactly where conditions change without you noticing. Opus 5.5 defaults to medium effort where Opus 5 defaulted to high, so anyone who switched models and left the setting alone is running a notch lower than before. Claude Code changed how subagents pick their effort level that same week. Opus 5.5's safeguards can hand a flagged request to an older model. And your own setup keeps growing, with more instructions, more memory and more tools every week. The LiveNerf README puts it plainly: "a changed harness looks exactly like a changed model."

Anthropic has been here before. In April it traced weeks of complaints to three changes in Claude Code, one of them a lower default effort, while the API itself was unaffected. Its own account says the early reports were "challenging to distinguish from normal variation in user feedback".

The best counterexample I've found is a GitHub issue from someone who scored their own sessions and found Opus 5.5's answers being corrected about five times as often after 19 October as in its first four weeks. That's an actual measurement and I take it seriously. But it counts corrections, and what gets corrected depends on what you're asking and how hard you're checking, both of which drift over a month of real work. It's the right kind of evidence aimed at a question it can't quite settle on its own.

My own fixed instrument

I'm not claiming to be above this. Back in July I built a model test suite and ran it against six models, so I own a dated baseline, albeit of now quite dated models. When I went to use it for this post, it turned out to be the wrong kind. It calls the model bare, with no system prompt, which is the opposite of the conditions anyone is complaining about. It ran through a routing service I no longer use, so I couldn't even hold the route steady. And it never tested Opus 5.5, which didn't exist yet. A fixed instrument, it turns out, has to fix the conditions as well as the questions.

The better evidence was in my personal assistant agent's logs. Mel, as it's called, runs a scheduled morning briefing for me most days, with the same instructions and tools at the same time. Between mid-October and today it ran that briefing on Opus 5.5 27 times, either side of Fable 5.2's release, and when I had it go through its own transcripts the numbers barely moved: about three minutes, around 16 model round trips, roughly 47 tool calls and typically no errors. This morning's run was in line with October. That measures how much work it does rather than how good the briefing is, but if Opus 5.5 had started cutting corners, that's where it would show first.

Then I did it myself

Which brings me to my own version of the same mistake. When Fable 5.2 arrived I switched Mel over to see what the difference was, and three days later I switched it back, convinced Fable was worse.

Because by the measures that matter to me, it was. I care about four things in an agent: how well it understands what I'm actually trying to do, how well it turns my vague hand-waving into what I meant, how well it gets my writing voice, and whether it sticks to the habits and corrections we've built up over time. On a job filling in a set of profile forms on a third-party site, Fable failed the first of those badly. I wanted help, and it went ahead and populated every field from my background notes, in words I wouldn't have used. It went gonzo on me, to borrow a word I last used about Opus 5. In my experience Opus has understood that on this kind of work we write it together. Once I switched back and explained what I wanted, the session went much more smoothly, with solid contributions rather than Fable's more out-there phrasing.

It was also hard on my subscription. On the last morning before I switched back, about two hours of work on Fable took my five-hour usage meter from roughly 10% to 50%. My weekly meter, which resets on Tuesdays and would normally be somewhere around 15% by Thursday morning, was at 41% when I checked after switching back to Opus, and a day and a bit of Opus work since has only taken it to 46%. Fable's list price per token is double Opus's, and Mel's figures put the routine briefing at about twice the cost per run, so a weekly meter moving at roughly twice its usual pace is about what you'd expect. Whether the five-hour meter tells a worse story I can't say, because I've never taken note of what a normal morning does to it.

Compared with what?

Now for the inconvenient part. When Mel went back through the transcripts, the picture was messier than my memory of it. On the routine work, Fable made the same number of tool calls in the same time with no more errors, needed about half as many round trips, and wrote more. Some of the phrasing I objected to on the forms came from Mel's own background notes, written while it was running on Opus. And when I switched back mid-session, Opus picked up Fable's assumption and started making edits until I stopped it and spelt out that I write and it helps. The model change and the change in how we were working happened within minutes of each other, so that session can't tell me which one fixed things.

So I've done exactly what I said the nerf crowd is doing: one operator, a few days, a setup built around one model, and a strong opinion. I'll stand by the opinion, because I'm the one paying, and Mel stays on Opus. What I can't tell you is that Fable 5.2 is a worse model, only that it was worse for me, in that setup, on the things I care about.

If you think your model has been nerfed, a few things are worth doing before you post about it:

• Check the effort setting and which model actually answered.
• Check what else changed: harness version, instructions, memory, and the way you've been asking.
• Find something you've run the same way many times and compare against that. If you have nothing, start now. A scheduled job that does the same work every day is a better baseline than a test suite you'll run once.

And if you're comparing two models, remember that your setup has had months to settle around one of them.