rooflineflip

null resultPython · llama.cppInference & servingJuly 11, 2026

the question

Does the backend's roofline regime flip the sign of speculative decoding?

what came out

On the M4, speculative decoding is a net loss for Q4_K_M models on both GPU and CPU (speedup 0.52-0.66, CIs exclude 1): quantization makes even the GPU compute-bound, erasing the verify-batch amortization spec-decode relies on. A verify_cost_ratio model predicts it within ~8%.

method & receipts

Result: null result — an honest negative, reported as it came out.
Pre-registered — the prediction was written down and committed before the run.

→ read the code and re-run it

github.com/v-code01/rooflineflip