Lost in the Prompt Order: Revealing the Limitations of Causal Attention in Language Models

arXiv:2601.14152v2 Announce Type: replace-cross Abstract: Large language models exhibit surprising sensitivity to the structure of the prompt, but the mechanisms underlying this sensitivity remain poorly understood. In this work, we conduct an in-depth investigation on a striking case: in...