hlascan

confirmedshipPython · HLA linear attentionModel behavior & reasoningJuly 23, 2026

the question

Is HLA's decayed second-order combine operator associative, so its chunk-parallel scan matches the serial recurrence?

what came out

HLA (arXiv:2510.27258) Theorem 4.1 claims its decayed second-order scan reproduces the serial recurrence, but the masked combine operator is not associative once gamma<1: the G,h cross summaries carry decay on the wrong operand. Balanced and serial folds agree at gamma=1 (7e-15) but diverge order-one at gamma=0.9 (2.3); closed-form gap matches to 9e-16.

method & receipts

Result: confirmed
Reproducible — one script re-runs the whole thing from scratch.
Tested — a correctness/benchmark suite ships alongside the code.

→ read the code and re-run it

github.com/v-code01/hlascan