From Tool Connection to Execution Control: Benchmarking Security Invariants in MCP-Style Agent Runtimes

The Security Gap in Agent Execution

The arXiv paper “From Tool Connection to Execution Control: Benchmarking Security Invariants in MCP-Style Agent Runtimes” addresses a critical blind spot in the rapidly evolving ecosystem of Model Context Protocol (MCP)-based agents. MCP, popularized by Anthropic, provides a standardized interface for connecting large language models (LLMs) to external tools, data sources, and prompts. However, the paper’s core finding is that the security model for these agents is dangerously fragmented once they move from connecting to executing.

What Happened

The researchers systematically benchmarked security invariants—properties that must hold true for safe operation—across multiple MCP-style agent runtimes. They found that while the connection layer (authentication, tool registration) is relatively well-defined, the execution layer (how agents actually invoke tools, manage state, and handle failures) lacks consistent security guarantees. Key issues include:

• Split security decisions: Different runtimes handle authorization, sandboxing, and resource isolation in ad hoc ways, creating gaps that malicious or misconfigured agents can exploit.
• No unified execution invariant: There is no standard for what constitutes a safe execution boundary—e.g., whether an agent can overwrite system files, access network resources beyond its scope, or persist state across sessions.
• Benchmarking gaps: Existing benchmarks focus on functional correctness (does the agent complete the task?) rather than security invariants (can the agent cause harm while completing the task?).

Why This Matters

This is not a theoretical concern. MCP-style agents are being deployed in production environments—coding assistants, data analysis pipelines, and automated customer service systems. If an agent can connect to a tool but then execute arbitrary code or access unauthorized data due to runtime-level security holes, the entire ecosystem becomes a liability.

The paper highlights a fundamental tension: MCP standardizes how agents connect, but not how they execute. This leaves runtime implementers (e.g., Claude Desktop, VS Code extensions, custom MCP servers) to invent their own security models, often inconsistently. For example, one runtime might sandbox file writes, while another relies on the agent’s own honesty—a recipe for disaster when agents are instructed to “use any tool necessary.”

Implications for AI Practitioners

• Audit runtimes, not just protocols: When adopting MCP, practitioners must evaluate the security invariants of the specific runtime, not just the protocol layer. A runtime that lacks execution sandboxing or privilege separation is a risk, even if it connects perfectly.

• Demand invariants, not features: The paper implicitly calls for a new class of benchmarks—security invariants that must hold across all MCP runtimes. Practitioners should push for standards like “agents cannot modify system files outside a designated workspace” or “agents cannot persist state beyond a single session without explicit user approval.”

• Design for least privilege: Even if runtimes improve, agents themselves should be designed with minimal tool access. The paper’s findings reinforce that an agent with a “delete file” tool and a weak runtime is a vulnerability waiting to happen.

Key Takeaways

• MCP standardizes tool connection but not execution security, leaving critical gaps in runtime-level sandboxing, authorization, and state isolation.
• Current benchmarks focus on agent functionality, not on security invariants—a blind spot that could lead to real-world exploits as agents are deployed in production.
• AI practitioners must audit runtime security models independently and advocate for standardized execution invariants across the MCP ecosystem.
• The split between connection and execution security means that even well-designed protocols can be undermined by poorly secured runtimes.