A Game Theoretic Free Energy Analysis of Higher Order Synergy in Attention Heads of Large Language Models

arXiv:2605.09515v1 Announce Type: new Abstract: Large language models rely on multihead attention, but interactions among heads remain poorly understood. We apply the Game Theoretic Free Energy Principle (GTFEP): a framework casting multiagent systems as distributed variational inference to analyze...