Series · 2 parts · 2,025–2,025

Mixture of Experts

How sparse expert models route a token, and what to do about the one expert that holds everyone else up.

How sparse mixture-of-experts really works: per-token routing, top-k gating, load balancing, and the memory bill you pay for the compute you save.

Mixture of Experts01

Why a mixture-of-experts layer runs at the speed of its busiest expert, how to measure it, and when predictive prefetching actually helps.

Mixture of Experts02