← Lab
determinism
mixedPython · llama.cppInference & servingJuly 6, 2026
the question
Is quantized llama.cpp inference bit-reproducible, and when isn't it?
what came out
Serial repeats are perfect (60/60 logit-identical), but concurrency breaks it: at 4 concurrent requests 0/8 are bit-identical -- only the sampled tokens match. Cross-batch and cache reuse perturb the logits too. So logprobs logged under load aren't reproducible, a hazard for calibration, perplexity, and reward scoring. Exact hashed-stream oracle with positive controls.
method & receipts
- Result: mixed
- Pre-registered — the prediction was written down and committed before the run.
- Reproducible — one script re-runs the whole thing from scratch.
- Independently reviewed — an adversarial review pass, kept in the repo.
- Machine-readable claims — every reported number, in a checkable file.
- Tested — a correctness/benchmark suite ships alongside the code.
→ read the code and re-run it
github.com/v-code01/determinism