Boundary Mass and the Soft-to-Hard Limit in Mixture-of-Experts

arXiv:2605.02124v1 Announce Type: cross Abstract: Softmax-routed mixture-of-experts models approach hard routing as the temperature tends to zero, but this limit is singular near routing ties. This paper studies that singularity at the population level for squared-loss MoE regression. The central...