clipcouple

confirmedshipPythonModel behavior & reasoningJuly 22, 2026

the question

How does global-norm gradient clipping propagate one tensor's gradient spike to the other parameters' updates?

what came out

A shared clip factor couples all tensors: one tensor spiking by S multiplies every healthy tensor's step by exactly 1/sqrt(1+(S^2-1)/L), approaching sqrt(L)/S. With L=12, a 50x spike drops every other layer to 0.069 of its step. Per-tensor clipping leaves them at 0.99.

method & receipts

Result: confirmed
Reproducible — one script re-runs the whole thing from scratch.
Tested — a correctness/benchmark suite ships alongside the code.

→ read the code and re-run it

github.com/v-code01/clipcouple