gqadict

the question

Does sharing one value matrix across a grouped-query head group lower the group's realized output rank relative to full multi-head attention?

what came out

Sharing V shrinks the group's value dictionary by exactly G (16=d_v vs 128=G*d_v at G=8), yet the concatenated output rank stays min(N, G*d_v)=128, equal to full MHA, because each head keeps an independent pattern and output projection. The cap bites only if the attention patterns are tied, collapsing rank to d_v=16.

→ read the code and re-run it