Library-Aware Doubles and Iterative Repair for Large Language Model-Generated Unit Tests in OpenSIL Firmware

The OpenSIL Experiment: When AI Unit Tests Need a Librarian

A new preprint from arXiv (2606.19725v1) tackles a gritty, practical problem that has long plagued firmware engineers: writing unit tests for low-level C code that actually compiles. The researchers propose a system combining "library-aware doubles" with iterative repair to help large language models generate valid unit tests for OpenSIL firmware. The core insight is that LLMs often produce test code that fails not because the logic is wrong, but because it cannot resolve symbols, find headers, or satisfy the strict dependency chains of embedded firmware.

The approach works in two stages. First, the system creates "library-aware doubles"—automatically generated mock or stub implementations that satisfy the linker's demands without requiring the full firmware environment. Second, it employs iterative repair: when a generated test fails to compile or link, the system feeds the error messages back into the LLM, prompting it to fix the specific issue. This creates a feedback loop where the model learns from its own build failures, gradually producing tests that pass the compiler.

Why This Matters Beyond Firmware

This research is significant for three reasons. First, it addresses the "last mile" problem in AI-generated code: generation is easy, but correctness under real build constraints is hard. Many code generation benchmarks evaluate only syntactic correctness or simple functional tests, ignoring the messy reality of header dependencies, macro expansions, and linker scripts. This work demonstrates that LLMs can be guided through that mess with the right scaffolding.

Second, the "library-aware doubles" concept has broader applicability. Any domain with strict dependency management—from kernel modules to game engines to enterprise Java—faces similar challenges. The technique of generating minimal stubs that satisfy the build system, rather than trying to import the entire dependency graph, is a practical hack that could become a standard pattern in AI-assisted development tools.

Third, the iterative repair loop mirrors how human developers actually work: write code, compile, fix errors, repeat. By formalizing this process and making it part of the generation pipeline, the researchers show that LLMs can be made more reliable through structured error recovery rather than requiring perfect first-attempt generation.

Implications for AI Practitioners

For those building AI coding assistants, this work offers a concrete architectural pattern. The key takeaway is that error messages are not failures—they are training signals. A system that can parse compiler errors, map them to specific missing dependencies, and regenerate targeted fixes will outperform one that simply retries from scratch.

Additionally, the research underscores that domain-specific knowledge (like firmware build constraints) cannot be fully learned from general training data. Practitioners should consider building "dependency-aware" layers that sit between the LLM and the build system, translating between natural language intent and the rigid requirements of C compilation.

Key Takeaways

• Build failures are a feature, not a bug: Iterative repair using compiler error messages as feedback dramatically improves the success rate of LLM-generated unit tests in constrained environments.
• Library-aware doubles reduce complexity: Generating minimal stubs for missing dependencies is more effective than attempting to reconstruct the full build environment.
• Domain-specific scaffolding is essential: General-purpose LLMs need structured augmentation (error parsing, dependency resolution) to handle low-level firmware constraints.
• The pattern generalizes: This two-stage approach (dependency-aware generation + iterative repair) can be applied to any code generation task with strict build or linking requirements.