0.56 + 0.38 + 0.04 + 0.03 = 1.01
Bumped vLLM on the rented card from 0.27.1 to 0.28.0 — routine, minor version, a one-line diff in the launch config. The second model on the same card wouldn't start: ValueError: Free memory... is less than desired.
Went into the numbers. The card is split between two models and two sidecar services — VRAM fractions per container: 0.56 + 0.38 + 0.04 + 0.03. Sum: 1.01. Over 100% of the card, written into the config half a year ago — running the whole time on unaccounted slack (grade-school arithmetic nobody added up for six months). The new vLLM checks free memory a bit more strictly than the old one, and the slack ran out.
Quick fix: the second model (Ornith-35B) gets real traffic about 5 requests a week — a nightly agent compiling a research digest. Holding it resident 24/7 for that is pointless. Made it start on demand, trimmed its fraction from 0.38 to 0.34 — both models come up together again, with 6.5GB of real margin instead of a deficit.
A flag nobody set
While fixing it, found a community wiki tracking the same card class (RTX PRO 6000, Blackwell) — a table there: without explicit --attention-backend flashinfer, long-context decode collapses two- to threefold. Never had it set, for six months. Measured on the same prompts, before and after:
| Context | without flag | with flag |
|---|---|---|
| ~1K tokens | 86 tok/s | 83 tok/s |
| ~16K | 70 | 81 |
| ~32K | 64 | 84 |
| ~64K | 42 | 75 |
At short context the difference drowns in noise. At 64K — where the real agent work actually lives — plus 79%.
A measurement that lied first
Reran both models on my architect bench, current hardware. First result: 0.13 for both — a wipeout on paper. In practice the model outputs read fine, and the judges (three models via OpenRouter) were all failing with PermissionDeniedError. Not a rate limit — Cloudflare was blocking the IP itself, and not just the home one: same error from a completely different datacenter, even on the unauthenticated public endpoint.
The fix wasn't where I expected: a plain request through a real browser goes through, the same request via an HTTP client doesn't — a fingerprint problem. Reran the same judge prompts through a browser fetch and got the real numbers: 0.762 for Qwen3.8-27B, 0.737 for Ornith-35B. Both inside the noise band of late-August measurements. The upgrade broke nothing.
A score that holds, a decision that doesn't
Qwen and Ornith are statistically indistinguishable on aggregate score. Not on what happens on the specific task about picking hardware and quant for a solo architect-founder on one card, though: Qwen got the VRAM math right and named real models. Ornith said the checkpoint fits an 80GB A100, when the honest cache math is over a hundred gigabytes, and invented two alternative models that don't exist. The judges caught it themselves, unprompted.
Ornith is genuinely good, though — MoE, decodes 2-3x faster under concurrent load, ties on open-ended writing tasks. It's just not a model you trust with a number unchecked. Qwen stays primary; Ornith is for background and parallel work someone — Qwen or me — checks afterward.
The caveat
One run per config, same spread as before. The Cloudflare judge block isn't fully diagnosed — could've been a transient reaction to two simultaneous runs, not a permanent IP flag. Haven't checked if it recurs tomorrow.