ropeabsorb

confirmedcertificatePython · PyTorchModel behavior & reasoningJuly 22, 2026

the question

Can RoPE be folded into MLA's weight-absorbed KV cache, or must the decoupled RoPE dimension stay separate?

what came out

Absorption makes the key matrix exactly rank d_c (4e-16 cliff), but the RoPE position-operator family spans exactly 2L = d_R dimensions (measured rank 16, 6e-16 cliff); a single absorbing matrix exists only with no RoPE (rank 1). Partial absorption is impossible, so the decoupled dimension must be exactly d_R.

method & receipts

Result: confirmed
Tested — a correctness/benchmark suite ships alongside the code.

→ read the code and re-run it

github.com/v-code01/ropeabsorb