ngeodesic

the question

Is nGPT's normalized-residual update a geodesic (SLERP) step on the sphere, or does it deviate by an exact amount?

what came out

It is a retraction, not a geodesic. With scalar alpha it lags SLERP by a gap that is zero only at alpha in {0, 1/2, 1} -- alpha=1/2 halves the angle exactly (4e-14 deg) -- growing to 49.6 deg at theta=170. With the real per-dimension alpha the update leaves the geodesic plane entirely (off-plane norm ~0.18).

→ read the code and re-run it