flashswamp

the question

Below what block size does a low-precision flash-attention softmax carry lose accuracy, and does the shipped Apple M4 GPU kernel stay above it?

what came out

A bf16 sequential softmax carry swamps: relative denominator error is 2.3% at n/Bc=256 and 73% at 8192, while a pairwise sum stays exact. The shipped M4 GPU attention kernel keeps bf16 error flat (5.85e-3 to 6.16e-3 from n=256 to 8192), showing it accumulates in fp32 and stays safe.

→ read the code and re-run it