Notes tagged
#research
4 posts
- 3 min read
The cheaper judge missed my own bar: 6.6% against 5%
Three judges grade my benchmark and cost three quarters of my API bill. I tested cheaper ones against rules written in advance. They did not pass, so the judges stay.
- 2 min read
The first table said 0.632. The real score was 0.845
Three closed models, 50 tasks, the same judges: the gap between them is smaller than the measurement's own noise.
- 3 min read
655 questions in one evening. A benchmark for my own model.
Built LII-Sport-Bench-RU v0.1 — 655 expert questions across 35 sports. Eleven AI agents in parallel. Methodology, not magic.
- 2 min read
Fine-tuned a model to GPT-5.4 quality for $330
$330, 16 hours on an H200, #9 of 30 on EduBench-RU. The 32B version I trained in parallel cost 3× the GPU and scored worse.