gramcliff

the question

Does the mini-batch used to speed up a TTT-Linear layer have a strictly smaller stability region than the online delta-rule layer it approximates?

what came out

Online steps are non-expansive iff eta*max||k||^2<=2 while the mini-batch chunk is stable iff eta*lambda_max(S)<=2, and since lambda_max(S)>=max||k||^2 the mini-batch region is a strict subset. At b=16 with correlated keys lambda_max(S)~15.5 gives a 15.5x shrink; an online-stable rate diverges (||W||~1e142) and the ceiling tightens like 2/b.

→ read the code and re-run it