sink-value

confirmedPythonInference & servingJuly 11, 2026

the question

Is the attention sink's value vector a content-free probability dump, or is it load-bearing?

what came out

Load-bearing. Zeroing the sink's value (key intact) raises Qwen2.5 perplexity +163-573%, specific to the sink and concentrated at layer 0 -- but via attention MASS, not a massive value norm. Exact oracle plus a code-disjoint verifier; adjudicates StreamingLLM vs Massive Activations.

method & receipts

Result: confirmed
Pre-registered — the prediction was written down and committed before the run.

→ read the code and re-run it

github.com/v-code01/sink-value