The Slowest Kid Problem: How a Super Captain Solves MoE's Biggest Headache
Why a mixture-of-experts layer runs at the speed of its busiest expert, how to measure it, and when predictive prefetching actually helps.
Mixture of Experts02
Subject · 2 posts
Why a mixture-of-experts layer runs at the speed of its busiest expert, how to measure it, and when predictive prefetching actually helps.
Mixture of Experts02
How sparse mixture-of-experts really works: per-token routing, top-k gating, load balancing, and the memory bill you pay for the compute you save.
Mixture of Experts01