← Lab
holblock
null resultPython · llama.cppInference & servingJuly 8, 2026
the question
Does a long generation head-of-line-block short requests on llama.cpp?
what came out
No -- continuous (iteration-level) batching inflates a concurrent 8-token request by a flat ~1.5x regardless of the long generation's length (128 vs 1024 tokens), versus the 11-52x it would wait if queued behind it. The measured case for iteration-level batching. Pre-registered, independently verified.
method & receipts
- Result: null result — an honest negative, reported as it came out.
- Pre-registered — the prediction was written down and committed before the run.
- Reproducible — one script re-runs the whole thing from scratch.
- Independently reviewed — an adversarial review pass, kept in the repo.
- Machine-readable claims — every reported number, in a checkable file.
- Tested — a correctness/benchmark suite ships alongside the code.
→ read the code and re-run it
github.com/v-code01/holblock