ELAS: Efficient Pre-Training of Low-Rank Large Language Models via 2:4 Activation Sparsity

arXiv:2605.03667v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved remarkable capabilities, but their immense computational demands during training remain a critical bottleneck for widespread adoption. Low-rank training has received attention in recent years due to its...