MoE fundamentals

From One Brain to Many: Understanding Mixture of Experts (MoE) Like You're 12

How sparse mixture-of-experts really works: per-token routing, top-k gating, load balancing, and the memory bill you pay for the compute you save.

Mixture of Experts01