Skip to main content
Vansh Verma
← Lab

penaltyledger

mixedupstream bugC++ · CPUInference & servingJuly 13, 2026

the question

Is the penalties sampler's O(1) token_count ledger exact, matching the histogram of its ring buffer across accept, reset, and clone?

what came out

Mixed. The accept and reset ledger is bit-exact against a from-scratch window histogram over 3,010,612 enumerated sequences (zero mismatches). But llama_sampler_penalties_clone copies prev without token_count, so a cloned sampler forgets its penalties and, after evicting a pre-clone token, drives a count negative and boosts a repeated token instead of penalizing it.

method & receipts

  • Result: mixed
  • Pre-registered — the prediction was written down and committed before the run.
  • Reproducible — one script re-runs the whole thing from scratch.
  • Tested — a correctness/benchmark suite ships alongside the code.

→ read the code and re-run it

github.com/v-code01/penaltyledger