MoE Expert Parallelism Deployment

Why it matters

EP is central MoE deployment. Understanding shapes MoE infra.

The architecture

Divide experts across GPUs.

Route tokens to expert GPUs.

Compute + return.

How it works end to end

EP size = num experts / experts per rank.

All-to-all: tokens to their experts.

Combine with TP + DP + PP.