IHBench: Evaluating Post-Interruption Recovery in Voice Agents with Structured Workflows

The Interruption Blind Spot in Voice AI

A new research paper from arXiv (2606.19595v1) introduces IHBench, a benchmark designed to evaluate how well voice agents handle interruptions during structured workflows. The research addresses a critical gap: while existing speech benchmarks measure accuracy or latency, they largely ignore the messy reality of human conversation—where users cut off the agent mid-sentence, change their mind, or provide partial information before being prompted.

IHBench tests voice agents across domains like customer service, healthcare scheduling, and account management, where multi-step procedures are the norm. The benchmark simulates common interruption types—such as correcting a previous answer, asking for clarification, or abruptly shifting the topic—and measures whether the agent can maintain procedural progress without losing context or requiring a full restart.

Why This Matters

Current voice AI systems, including those powering phone trees, virtual assistants, and customer support bots, are brittle when interrupted. A user saying “Actually, I meant Tuesday, not Wednesday” mid-flow often forces the system to reset the entire booking process. This isn’t just an inconvenience—it creates friction that drives users to human agents, undermining the cost and efficiency benefits of automation.

The research highlights a fundamental design flaw: most voice agents treat conversations as linear, turn-based transactions. Real human conversations are nonlinear, filled with overlaps, corrections, and tangents. IHBench provides a structured way to quantify this gap, which has been largely anecdotal until now.

For AI practitioners, the implications are immediate. If your voice agent cannot recover from a simple interruption like “No, that’s wrong—let me start over,” it will fail in production environments where users are impatient, distracted, or multitasking. The benchmark also exposes a deeper challenge: maintaining state across interruptions requires not just better speech recognition, but smarter dialogue management and memory systems.

Implications for AI Practitioners

First, interruption handling is not a speech recognition problem—it’s a system architecture problem. Even perfect ASR will fail if the dialogue manager cannot update its internal state mid-stream. Practitioners should audit their voice pipelines for state persistence across turns.

Second, structured workflows need explicit interruption recovery logic. Most current systems handle interruptions as errors, triggering fallback prompts or resets. IHBench suggests a better approach: treat interruptions as first-class conversational events, with dedicated recovery paths that preserve partial progress.

Third, benchmarking must evolve beyond accuracy metrics. IHBench’s focus on procedural continuity—can the agent complete the task despite interruptions?—is more aligned with real-world utility than word-error-rate or response-time metrics. Teams should develop their own interruption test suites tailored to their specific workflows.

Finally, this research signals a shift toward human-centered evaluation. As voice AI moves from simple Q&A to complex task completion, benchmarks must reflect the chaotic, cooperative nature of human conversation. IHBench is a step in that direction, but practitioners should expect more such benchmarks to emerge, particularly for multimodal and real-time interactions.

Key Takeaways

• IHBench fills a critical gap by testing voice agents on interruption recovery during structured workflows, not just on accuracy or latency.
• Current voice AI systems are brittle when interrupted, often requiring full workflow resets—a major barrier to real-world adoption.
• Practitioners need to design dialogue managers with explicit interruption recovery logic, not just better speech recognition.
• The benchmark underscores the need for evaluation metrics that measure task completion under realistic, messy conversational conditions.