This repository presents a comprehensive survey of agent execution harnesses for Large Language Model (LLM) agents, addressing the critical, yet often overlooked, role of the harness architecture in determining agent reliability and performance at scale. It targets researchers designing agent infrastructure and practitioners building production systems, offering a novel taxonomy and detailed analysis of over 23 existing systems and 110+ papers. The survey provides a structured framework for understanding why harness design is a binding constraint on deployed agent reliability and highlights key technical challenges that impede broader adoption.

How It Works

The core contribution is the formalization of the agent execution harness as a six-component architectural object: H = (E, T, C, S, L, V), representing the Execution Loop, Tool Registry, Context Manager, State Store, Lifecycle Hooks, and Evaluation Interface. This framework elevates the harness from implicit infrastructure to an explicit research target, challenging the dominant narrative that agent performance is solely determined by the underlying LLM. Empirical evidence from industry deployments like Pi Research, OpenAI Codex, and Stripe Minions is cited to underscore that harness design is a critical determinant of agent reliability, often more so than the model itself.

Quick Start & Requirements

This repository is a survey and academic resource, not a software project. It does not provide installation instructions, runnable code, or system requirements.

Highlighted Details

• Formalizes the agent harness with six key components (E, T, C, S, L, V), providing a structured and comprehensive framework for analyzing and comparing agent systems.
• Analyzes over 110 papers and 23 systems, introducing the Harness Completeness Matrix for systematic comparison across diverse agent architectures and identifying which components are implemented by each.
• Presents compelling empirical evidence demonstrating the significant impact of harness design on agent reliability, citing industry examples where harness improvements drastically boosted performance without model changes.
• Identifies and details nine open technical challenges in agent harness development, including formal security models, cross-harness portability, protocol interoperability, context economics, Byzantine fault tolerance, and compositional verification, crucial for advancing production-grade systems.

Maintenance & Community

The project is actively maintained, with contact information provided for suggestions and corrections. Links to the paper, preprints version (v3), and DOI are available for further reference, facilitating engagement and updates. The survey is under active development, with a full manuscript pending release.

Licensing & Compatibility

The project is licensed under CC-BY-4.0, a permissive license that allows for broad use, distribution, and adaptation, including for commercial purposes, provided proper attribution is given.

Limitations & Caveats

As a survey, this repository does not offer a deployable agent harness or code. It focuses on analyzing existing research and systems, identifying critical gaps where production-grade infrastructure is still lacking. The full manuscript is under active development, with some cited works being preprints, indicating the field is rapidly evolving.