moeproxy

the question

Does the standard MoE load-balancing loss control the hard top-1 expert load, or only the soft router marginal?

what came out

The exact identity L=1+N<f-u,P-u> shows the loss is an alignment of hard-load and soft-marginal imbalance, not a magnitude. Its minimum L=1 holds for any hard load paired with a uniform marginal, and the detached f kills the gradient there. A router driven into the blind region reads L~1.000 while dropping 26.4% of tokens at N=64.

→ read the code and re-run it