Send the Poets Home: Expert Pruning, a Student Model, and When to Pick Each
A bank rents the biggest MoE around, salary day brings a thousand customers, and the team learns what to prune, why Bangla breaks, and when to teach a student.
Mixture of Experts03