Scaling Multi-Node Mixture-of-Experts Inference Using Expert Activation Patterns

arXiv:2604.23150v1 Announce Type: cross Abstract: Most recent state-of-the-art (SOTA) large language models (LLMs) use Mixture-of-Experts (MoE) architectures to scale model capacity without proportional per-token compute, enabling higher-quality outputs at manageable serving costs. However, MoE...