straggler problem

The Slowest Kid Problem: How a Super Captain Solves MoE's Biggest Headache

Why a mixture-of-experts layer runs at the speed of its busiest expert, how to measure it, and when predictive prefetching actually helps.

Mixture of Experts02