residwrite

the question

Do transformer blocks add to or erase the residual stream, and does a late erase collapse the residual norm?

what came out

Qwen2.5's dominant sink channel is built up through the body and torn down by the final MLP, which writes against the channel's own sign while attention barely touches it. That one channel accounts for the apparent norm collapse: removing it, the final block's norm effect flips from x0.83 to x1.27 at 0.5B. Holds across 0.5B/1.5B/3B.

→ read the code and re-run it