nopealias

confirmedshipPython · NumPyModel behavior & reasoningJuly 24, 2026

the question

How does the reciprocal position code that NoPE recovers behave under low-precision rounding?

what came out

NoPE's softmax normalization forces the recovered position code to be exactly `1/t`, whose adjacent gap `1/(t(t+1))` decays quadratically. Neighbouring positions round to identical bits at `t* ~ 2^(m+1/2)` for an m-bit mantissa: t*=191 (bf16), 1465 (fp16), fp32 immune. A mantissa-width law, not an fp32-cast artifact.

method & receipts

Result: confirmed
Reproducible — one script re-runs the whole thing from scratch.
Tested — a correctness/benchmark suite ships alongside the code.

→ read the code and re-run it

github.com/v-code01/nopealias