On Semantic Loss Fine-Tuning Approach for Preventing Model Collapse in Causal Reasoning

arXiv:2605.05438v1 Announce Type: cross Abstract: Standard fine-tuning of transformer models on causal reasoning tasks leads to catastrophic model collapse, where models learn trivial solutions such as always predicting "Yes" or "No" regardless of input structure. We demonstrate that fine-tuning...