← Lab
covfirewall
confirmedshipPython · PyTorchModel behavior & reasoningJuly 22, 2026
the question
Is the mLSTM readout clamp a divide-by-zero guard, or does it serve a different numerical role?
what came out
The normalizer S=n.q is a signed sum with unbounded condition number mass/|S| (softmax is exactly 1), negative on about half of steps. The clamp max{|S|,1} is a cancellation firewall capping a 1/|S| blow-up (readout norm ~10 versus ~62000), active exactly on the worst-conditioned steps and the default regime at typical scale.
method & receipts
- Result: confirmed
- Tested — a correctness/benchmark suite ships alongside the code.
→ read the code and re-run it
github.com/v-code01/covfirewall