Caracal: Causal Architecture via Spectral Mixing

arXiv:2605.00292v1 Announce Type: cross Abstract: The scalability of Large Language Models to long sequences is hindered by the quadratic cost of attention and the limitations of positional encodings. To address these, we introduce Caracal, a novel architecture that replaces attention with a...