Repetition over Diversity: High-Signal Data Filtering for Sample-Efficient German Language Modeling

arXiv:2604.28075v1 Announce Type: cross Abstract: Recent research has shown that filtering massive English web corpora into high-quality subsets significantly improves training efficiency. However, for high-resource non-English languages like German, French, or Japanese, aggressive filtering...