Skip to main content
Vansh Verma
← Lab

parallelscale

mixedPython · llama.cppInference & servingJuly 6, 2026

the question

Does adding parallel slots raise throughput, or just split the context?

what came out

Throughput doesn't monotonically saturate -- it plateaus then jumps (prediction 1 falsified). On the 1.5B: +33% at 2 slots, then only +3% at 4 and 8; the 0.5B keeps gaining (+64/+35/+11%). More slots also split the per-slot context (4096->1024). Predictions 2-4 held.

method & receipts

  • Result: mixed
  • Pre-registered — the prediction was written down and committed before the run.
  • Reproducible — one script re-runs the whole thing from scratch.
  • Independently reviewed — an adversarial review pass, kept in the repo.
  • Machine-readable claims — every reported number, in a checkable file.
  • Tested — a correctness/benchmark suite ships alongside the code.

→ read the code and re-run it

github.com/v-code01/parallelscale