Agents That Know Too Much: A Data-Centric Survey of Privacy in LLM Agents

What Happened

A new arXiv survey (2606.26627) systematically examines privacy vulnerabilities in LLM agents—systems that don't just generate text but actively query databases, search documents, call external APIs, retain conversation histories, and execute actions on behalf of users. The paper takes a "data-centric" approach, shifting focus from model-level privacy risks (like memorization) to the data flows and operational contexts where privacy breaches actually occur. It catalogs how agents accumulate sensitive information across multiple channels: through persistent memory of past interactions, through API calls that expose user credentials or proprietary data, and through tool-use patterns that can leak behavioral profiles.

Why It Matters

This survey arrives at a critical inflection point. The industry is rapidly moving from stateless chatbots toward stateful agents that maintain context, build user profiles, and execute multi-step workflows. Each new capability—tool use, memory persistence, autonomous action—introduces a corresponding privacy surface. The paper's key insight is that these risks are not merely theoretical or reducible to "the model leaked training data." Instead, they are structural: an agent that remembers your last ten queries about medical symptoms has created a longitudinal health record, even if no single query was sensitive. Similarly, an agent that calls a flight booking API with your credit card details has created a transactional privacy risk that no amount of model alignment can fix.

For practitioners, this reframes the privacy problem. The dominant narrative has been about preventing models from regurgitating training data or responding to prompt injection. This survey argues that the more pressing risks come from the agent's operational architecture—how data flows between the user, the agent's memory, and external services. A well-aligned model can still cause a privacy breach if its tool-calling logic is poorly scoped.

Implications for AI Practitioners

First, memory management becomes a privacy engineering problem. Developers building agents with persistent memory need to implement data retention policies, user-controlled deletion, and clear audit trails—not just for compliance but because accumulated context can become a liability. Second, API access control needs granularity. Agents that call external services should do so with least-privilege credentials, and those credentials should be scoped to the minimum necessary action, not the agent's full authority. Third, privacy impact assessments must cover the agent's entire data lifecycle—from input collection through internal state storage to output and tool responses. The survey implicitly warns against treating privacy as a model-level checkbox.

Key Takeaways

• LLM agents create privacy risks primarily through their operational architecture (memory, tool use, API calls) rather than through model memorization alone.
• Persistent agent memory transforms ephemeral conversations into longitudinal behavioral profiles, requiring explicit data governance policies.
• Tool-calling capabilities introduce transactional privacy risks that demand least-privilege access controls and scoped credentials.
• Privacy engineering for agents must cover the full data lifecycle—input, internal state, output, and third-party service interactions—not just model alignment.