Skip to main content
← Lab

covfirewall

confirmedshipPython · PyTorchModel behavior & reasoningJuly 22, 2026

the question

Is the mLSTM readout clamp a divide-by-zero guard, or does it serve a different numerical role?

what came out

The normalizer S=n.q is a signed sum with unbounded condition number mass/|S| (softmax is exactly 1), negative on about half of steps. The clamp max{|S|,1} is a cancellation firewall capping a 1/|S| blow-up (readout norm ~10 versus ~62000), active exactly on the worst-conditioned steps and the default regime at typical scale.

method & receipts

  • Result: confirmed
  • Tested — a correctness/benchmark suite ships alongside the code.

→ read the code and re-run it

github.com/v-code01/covfirewall