Agentic Publication Protocol: An Attempt to Modernize Scientific Publication

The Static Manuscript Problem

A new preprint on arXiv proposes an "Agentic Publication Protocol" that aims to drag scientific publishing out of the 17th century. The core observation is correct: modern science produces dynamic, computational workflows—code, datasets, interactive figures, and reproducibility checks—yet the primary output remains a static PDF or HTML document. The protocol envisions a system where publications are not just read but executed, with AI agents capable of reproducing results, testing edge cases, and even suggesting follow-up experiments.

Why This Matters

The disconnect between how science is done and how it is published has real costs. Reproducibility crises in fields like psychology, medicine, and machine learning often stem from insufficient documentation of tacit knowledge—the "how" behind the "what." A static paper cannot convey the exact environment, parameter choices, or failure modes encountered during research. The Agentic Publication Protocol attempts to encode this tacit knowledge into machine-readable, executable formats.

For AI researchers specifically, the problem is acute. Benchmarks, model training configurations, and evaluation pipelines are notoriously underdocumented. A paper claiming state-of-the-art results may omit the precise random seed, hardware-specific optimizations, or data preprocessing steps that made the result possible. An agentic protocol could, in theory, allow a reviewer or practitioner to run the exact pipeline, verify the claims, and explore sensitivity to changes.

Implications for AI Practitioners

If adopted, this protocol would shift the role of the publication from a static record to a living artifact. AI practitioners would need to think of their papers as software packages with versioned dependencies, test suites, and execution traces. This raises practical challenges: who maintains the code after publication? How do we handle broken dependencies as libraries evolve? The protocol must address long-term preservation, not just initial reproducibility.

There is also a tension between openness and control. An agentic system that can execute arbitrary code from a paper introduces security risks. Malicious actors could embed harmful operations in ostensibly benign research. The protocol must include sandboxing and provenance tracking to mitigate this.

For the AI community, the most immediate benefit would be in peer review. Reviewers could run experiments, verify claims, and test robustness without relying on the authors' goodwill. This could reduce the prevalence of irreproducible results and accelerate the cycle of scientific validation.

Key Takeaways

• The Agentic Publication Protocol proposes replacing static manuscripts with executable, AI-readable artifacts that encode tacit knowledge about code, data, and workflows.
• It directly addresses the reproducibility crisis in computational sciences by making the "how" of research as accessible as the "what."
• AI practitioners will need to adopt software engineering best practices—versioning, dependency management, and test automation—for their publications.
• Security, maintenance, and long-term preservation remain open challenges that the protocol must solve to gain widespread adoption.