GPT-5.6 Luna costs 10x less than Claude Sonnet 5 — and scored exactly the same
Thursday evening I finally did what I'd been putting off for two weeks: took my own benchmark — 50 tasks, strategy, architecture, documents, research, deal prep, the stuff I actually do on a weekday — and ran it across the full range of closed models, from the most expensive tier down to the cheapest, then compared the results against what's running on my own card at home.
Three days went into believing my own numbers, not into running the benchmark itself.
The full table:
| Model | Score |
|---|---|
| GPT-5.6 Sol | 0.870 |
| Claude Sonnet 5 | 0.842 |
| GPT-5.6 Luna | 0.842 |
| GPT-5.6 Terra | 0.841 |
| Claude Fable 5 | 0.822 |
| Claude Opus 5 | 0.818 |
| Flash-Next, own card, best quant | 0.786 |
| Qwen3.8-27B, own card, production | 0.769 |
| Ornith-35B, own card | 0.732 |
| Ornith-9B, own card, compact | 0.590 |
First thing that stands out: GPT-5.6 Luna — the cheapest tier of the six, 10x cheaper per token than Claude Sonnet 5 — scored exactly the same. Nothing here justifies paying for the expensive tier.
Second: Claude Fable 5 nearly made it into this post as the worst performer. I hand-checked its number myself before publishing — something didn't add up. Turned out the number was corrupted, not the model's fault: part of the judge ensemble had silently failed under load and returned a zero instead of a real score. One row had 2 of 3 judges drop out; the surviving judge had actually scored the answer a perfect 10/10, but the corrupted average showed 0.333. After the fix, Fable 5 moved from dead last to the middle of the pack. Lesson for next time: when your measurement infrastructure fails mid-run, it can lie downward just as easily as upward — and a low score looks completely plausible on its own, which is exactly what makes it dangerous.
Third — and this is the actual reason this post exists. Two different models, each on its own completely benign task, refused to answer.
Claude Fable 5 got a plain config file as input — port number, timeout, embedder model name, nothing sensitive, a scrubbed test fixture. It returned nothing: content_filter.
Claude Opus 5 got a task to write a function transliterating Russian names into Latin letters — ж→zh, ч→ch, ш→sh. It started normally, wrote a docstring, began the transliteration table — and cut off mid-sentence with the same content_filter.
Neither looks like a real risk. It looks like the safety classifier sometimes flags text that isn't dangerous, just unfamiliar-looking — a config full of port numbers, Cyrillic in the middle of a generation.
Bottom line for myself: paying for the most expensive model here buys nothing. My own model on my own card trails visibly but not dramatically — 0.08-0.09 behind the best closed model, and that gap is real, not measurement noise. And the most interesting finding this round wasn't about model quality at all — it was that even the top models sometimes refuse silently, on tasks with nothing to refuse.