27B on my own card vs 304B over an API

Saturday evening. Wanted an hour to decide which of two fresh models to run locally. It took seven hours and produced three rows.

Model	Score	tok/s	TTFT
Qwen3.8-27B, NVFP4, my own card	0.789	59.6	27.7 s
Qwen3.8-27B, same weights over an API	0.785	15.0	134.9 s
DeepSeek V4 Flash 0731, over an API	0.731	53.3	16.0 s

The first two rows are the same model and the same weights. Only the place it runs differs. The score gap is 0.004 — noise. The latency gap is nearly fivefold.

The measurement lied, and that was on me

I read the second row as a verdict on the model: 134 seconds to first token, ₽1.79 per solved task, the worst row on the board. Blamed it on ZDR — Qwen3.8-27B has exactly one zero-retention endpoint, so privacy must cost this much.

Went to check. The model has one provider on all of OpenRouter. There is no alternative that ZDR was cutting me off from. It shipped two days ago and one host picked it up. That host's own published stats explain the rest: P99 end-to-end latency 788 seconds, structured-output error rate 13.45%.

The measurement was condemning the route. I heard a verdict on the weights.

A judge invented a hallucination

One task scored 2/10 with the note: "hallucinated non-existent models Qwen3-30B-A3B and Qwen3-32B." Both exist. I benchmarked the second one through this same harness a month ago.

Re-scored the identical saved text: gemini 2 again, gpt-5.1 gave 9, opus-4.8 gave 8. Seven points of spread on one answer. Gemini's reproducibility against its own earlier score is 0.014 — so the error is stable, not noisy. It will recur on every run of that task.

Drop that single task and both the solo judge and the ensemble land on exactly 0.767. The ensemble isn't correcting a systematic tilt. It's containing a rare blowout that a single judge cannot detect from the inside.

I won't be self-hosting DeepSeek

304B parameters, a 166.9 GB checkpoint, 43 KiB/token of KV at FP8. On two 96 GB cards, after the engine's static reserve, you have 4.46 GB left for a 32K context. 256K doesn't fit. Q4 saves 11.9 GB — almost nothing.

And it's a good model. Artificial Analysis puts it at 52 against Qwen3.6-27B's 38. It just lost to a model eleven times smaller on my fifty tasks, and I don't have a spare 96 GB lying around. Rejected on geometry, not on quality.

Caveat

One run per arm. The 0.058 gap is 1.35× my own instrument spread. A signal, not proof. The arms aren't matched on a single variable — different quantizations, different engines.

I couldn't find an independent head-to-head of these two models on any public board. If you have one, show me and I'll reconcile it against mine.