mpsrecompile

confirmedshipPython · Apple M4Inference & servingJuly 12, 2026

the question

Does the Apple M4 MPS backend recompile a Metal pipeline for every new tensor shape?

what came out

The first call at a never-seen shape costs a median 20-39x the steady per-call time (matmul 20x, softmax 39x, elementwise 34x), one-time and cached per shape. It is keyed on shape not size: two shapes of equal element count both pay. Larger matmuls reach 37-467x.

method & receipts

Result: confirmed
Pre-registered — the prediction was written down and committed before the run.
Reproducible — one script re-runs the whole thing from scratch.

→ read the code and re-run it

github.com/v-code01/mpsrecompile