Skip to main content
Vansh Verma
← Lab

constraintcost

null resultPython · llama.cppInference & servingJuly 6, 2026

the question

Is grammar-constrained (JSON-schema) decoding actually slower per token?

what came out

Barely -- both pre-registered predictions were falsified. At matched token count the tax is ~1.02-1.04x for medium and complex grammars (only a simple grammar that closes early shows 1.79x). Constrained decoding is nearly free per token; the cost people fear isn't there.

method & receipts

  • Result: null result — an honest negative, reported as it came out.
  • Pre-registered — the prediction was written down and committed before the run.
  • Reproducible — one script re-runs the whole thing from scratch.
  • Independently reviewed — an adversarial review pass, kept in the repo.
  • Machine-readable claims — every reported number, in a checkable file.
  • Tested — a correctness/benchmark suite ships alongside the code.

→ read the code and re-run it

github.com/v-code01/constraintcost