Skip to main content
← Lab

frozenrank

confirmedshipPython · PyTorchModel behavior & reasoningJuly 22, 2026

the question

Does DeltaProduct's product of n_h Householder factors bound its per-token forgetting rank and pin its spectral radius regardless of the beta values?

what came out

For n_h<d the transition freezes a (d-n_h)-dim subspace and pins rho=1 for any beta (|rho-1|<=3e-15), with rank(A-I)<=n_h; contraction below rho=1 needs full rank n_h>=d. At the reflection setting minimal n_h equals rank(g-I), so DeltaNet (n_h=1) cannot rotate at all, staying Frobenius-distance exactly 2 from any 2-plane rotation.

method & receipts

  • Result: confirmed
  • Tested — a correctness/benchmark suite ships alongside the code.

→ read the code and re-run it

github.com/v-code01/frozenrank