Notes tagged
#llm
19 posts
- 2 min read
29 Hours of Silent Maximum-Depth Reasoning
Set the model to think at a moderate level. It quietly ran at full depth for over a day instead — no error, just silence.
- 3 min read
The Host Froze the First Night. Two Days Later, a Private AI That Saves ₽75,500 a Month
Bought a used GPU — the host froze the first night. Two days later: five real fixes, from power draw to the finance model.
- 3 min read
0.56 + 0.38 + 0.04 + 0.03 = 1.01
The VRAM budget for two models on one card never added up — a routine vLLM upgrade finally proved it.
- 3 min read
GPT-5.6 Luna costs 10x less than Claude Sonnet 5 — and scored exactly the same
Ran my own benchmark against six closed models and four local ones — and found two cases where the top models silently refused completely benign tasks.
- 4 min read
Qwen3.8-Flash-Next vs production 27B across three runs: 0.789 was the wrong config
New Qwen4-preview architecture against the production 27B across three runs each — and why my last post cited the wrong config.
- 3 min read
2-bit DeepSeek, Ornith-35B, and a bug in my own measurement
Three models I hadn't benchmarked before, and a scoring bug someone else caught, not me.
- 3 min read
18 quants on one card: 3x faster doesn't mean smarter
Ran 18 configurations across three models on a rented card. Got a sizing table for any VRAM budget — and one defect the vendor benchmark never showed.
- 5 min read
More memory doesn't mean faster: prefill, decode, and the Apple M5 Ultra launch
Apple shipped M6 and M5 Ultra. Working through the roofline model and real numbers to explain why prefill and decode hit different hardware walls — and who actually needs 512GB of unified memory.
- 2 min read
Claude Enterprise: the admin sees everything, deleted chats included
A Reddit thread confirmed it: Claude Enterprise admins can read an employee's full chat history, deleted and incognito included. The question isn't the feature — it's the architecture.
- 3 min read
27B on my own card vs 304B over an API
50 tasks, three judges, two models. The smaller one won — but first I misread my own measurement.
- 5 min read
Third-party NVFP4 vs official FP8 on Qwen3.8-27B: 1.35× faster
My NVFP4 checkpoint (unsloth) beats official FP8 by 1.35× on Qwen3.8-27B, backend-controlled. Quality is still an open question, and I caught myself overclaiming twice along the way — once about hardware compatibility, once about a comparison model.
- 2 min read
Claude's watermarks: a signal, not proof
Anthropic will mark Claude's text with invisible watermarks. What a mark actually tells you — and why the real question is the contour, not the mark.
- 1 min read
One model, two circuits: own iron and cloud
We're a Yandex Cloud partner now. One Qwen3.6, two delivery circuits — own iron at Selectel and cloud via the Yandex Cloud API. Both 152-FZ-attested.
- 5 min read
Russian open models on my own Blackwell — and where NVFP4 breaks
An open RU-model ladder on an RTX PRO 6000: the dense 27B takes the top, a 12B lands half a step behind at half the size, and NVFP4 on Blackwell came up for only one of three.
- 2 min read
Fixed NVFP4 on Blackwell — it was missing CUDA 12.9
Two Blackwell bugs turned out to be one CUDA-toolkit gap, not a vLLM bug. Both models run NVFP4 now; memory ran 4-5GB heavier than the morning estimate.
- 3 min read
DeepSeek V4 Flash on my architect bench: 0.717 and ₽0.022 per solved task
DeepSeek V4 Flash scored 0.717 on my architect bench: below the top-3, above V4-Pro at 4.8× lower price — and for the first time realistic on 2×RTX PRO 6000.
- 2 min read
The boring bet: the frontier arrived where I've been building
I bet on verifiable AI — grounded in the client's corpus, cited, with an honest refusal — back when it sounded niche. I went through dozens of the year's biggest AI interviews: the frontier describes exactly that bet. And I have the one thing the voices on stage don't — a number.
- 2 min read
The full turn: the model is an API call
The full turn: the model is an API call for kopecks. Two circuits — public on CPU, sovereign on-site. The edge lives in the layer above the model.
- 2 min read
Picking the model for the hardware I have
Picking a model for the trusted-AI build on 24 GB. The bench beat intuition: a MoE beat a dense model twice its size; 'thinking' only got in the way.