From Signals to Structure: How Memory Architecture Drives Language Emergence in LLM Agents

This paper, posted on arXiv, revisits a classic question in cognitive science and linguistics: how do agents invent a shared language from scratch? The researchers use a Lewis signaling game, where a sender and receiver must coordinate on a code using only their interaction history, but they replace traditional rule-based agents with large language models (LLMs). The critical variable is the memory architecture—how each agent stores and recalls past interactions. The study tests five different memory designs across various communication channel constraints (e.g., limited vocabulary, noisy channels).

The core finding is that the structure of memory directly dictates the type of language that emerges. For instance, agents with a simple, unbounded memory tend to develop overly verbose or redundant codes, while those with a compressed, bottlenecked memory converge on more efficient, compositional structures—closer to what we might call a “grammar.” This suggests that the constraints on an agent’s ability to remember and retrieve past signals are not a bug, but a feature that drives the emergence of systematic language.

Why It Matters

This research is significant because it moves beyond the “black box” view of LLMs. Most current work treats LLMs as monolithic reasoning engines; this paper treats them as agents embedded in a dynamic, interactive environment. It demonstrates that the architectural choices we make—specifically around memory—have a profound, causal effect on the emergent behavior of multi-agent systems. For AI practitioners, this is a wake-up call: the performance of a multi-agent system is not just a function of the base model’s intelligence, but of the interaction protocols and memory mechanisms you design.

The study also provides a concrete, empirical link between the constraints of human cognitive architecture (e.g., working memory limits) and the emergence of structured language. This lends support to the hypothesis that the structure of human language is not arbitrary, but is a direct consequence of our cognitive limitations.

Implications for AI Practitioners

• Memory is a design parameter, not an afterthought. When building multi-agent LLM systems (e.g., for automated negotiation, collaborative coding, or simulation), you must deliberately design the memory architecture. A naive “store everything” approach can lead to inefficient or chaotic communication. A constrained memory can force agents to be more precise and structured.

• Channel constraints matter. The paper shows that the communication channel (e.g., limited token budget, noisy transmission) interacts with memory architecture. Practitioners should test their systems under realistic channel constraints, not just in ideal, high-bandwidth settings. The emergent language may break down or shift dramatically when the channel is tightened.

• Emergent language is not always optimal. Just because two LLM agents develop a shared code does not mean it is the most efficient or interpretable code. The paper’s results suggest that you can steer the emergent language toward desirable properties (e.g., compositionality, conciseness) by tuning the memory architecture and channel parameters.

Key Takeaways

• The memory architecture of LLM agents directly shapes the structure and efficiency of the language they invent from scratch.
• Constrained memory (e.g., compressed or limited recall) drives the emergence of more compositional and efficient codes, similar to human language.
• AI practitioners must treat memory as a critical design variable in multi-agent systems, not a default setting.
• Testing under realistic channel constraints (noise, token limits) is essential to understand how emergent communication will behave in production.