mpsaccum

confirmedshipPython · Apple M4Inference & servingJuly 12, 2026

the question

Does the M4 GPU fp16 matmul accumulate in fp16 or fp32, and does fp16 deliver 2x throughput?

what came out

It accumulates in fp32. The MPS fp16 error sits at 3.6e-4, flat in K and 72 sigma below the best-case pairwise fp16 accumulator floor, ruling out fp16 and tf32. fp16 throughput is only 1.10 to 1.12x fp32 at large sizes, nowhere near the folk 2x.

method & receipts

Result: confirmed
Pre-registered — the prediction was written down and committed before the run.
Reproducible — one script re-runs the whole thing from scratch.
Tested — a correctness/benchmark suite ships alongside the code.

→ read the code and re-run it

github.com/v-code01/mpsaccum