Why it matters

Expert choice fixes load balance. Growing use.

Advertisement

The architecture

Experts pick top-k tokens.

Load balance automatic.

No dropped tokens.

Expert choice routingExperts picktop-k tokensLoad balanceautomaticNo dropsguaranteedZhou et al. 2022; Google implementation; growing production use
Expert choice routing.
Advertisement

How it works end to end

Zhou 2022.

Automatic balance.

Growing production use.