deltafade

the question

At what exact rate does DeltaNet forget old associations, and how do Gated DeltaNet's two forgetting channels combine?

what came out

For isotropic keys the mean-square retention rate is exactly beta(2-beta)/d (measured 0.996016 at beta=0.3, d=128 over four million keys), not the mean-field beta/d, giving a horizon tau=d/(beta(2-beta)) linear in state dim. The explicit gate and implicit delta rule multiply exactly; the operator is non-expansive on beta in [0,2].

→ read the code and re-run it