<2-3 sentences summarising what the project addresses and solves, the target audience, and the benefit.> ScholarLoop is an autonomous AI research system designed to emulate a PhD's workflow, from literature review to experimental write-up, on a single-GPU budget. It employs a multi-agent loop with deterministic guards against reward-hacking and hallucination, enabling reproducible, honest AI-driven scientific discovery.

How It Works

The system orchestrates eight specialized agents (Director, Lit Scout, Reasoner, Debate Panel, Funnel, Runner, Reflector, Advisor, Writer/Reviewer) in a structured loop. It begins by scouting literature and identifying research gaps, then proposes experiments grounded in prior work. A budget-aware funnel screens ideas cheaply before running real PyTorch experiments. Crucially, a deterministic harness validates agent claims against ground truth, distills lessons into a decaying skill library, and ensures the loop cannot be reward-hacked, treating the outer loop engineering as the core product.

Quick Start & Requirements

• Installation: pip install -e ".[dev]" for core functionality (PyYAML, jsonschema, pytest). pip install -e ".[dev,engines]" adds PyTorch and scikit-learn for real experiments. pytest -q ".[llm]" adds the Anthropic client for live API runs.
• Prerequisites: Python 3.10+. PyTorch and scikit-learn are needed for real experiments. An Anthropic API key is required for live runs with Claude Opus.
• Resource Footprint: Designed for a single-GPU budget; core loop runs deterministically without GPU or API keys using MockLLM.
• Links:
  • Quickstart example: examples/quickstart.py
  • Campaign demo: examples/campaign_demo.py
  • Live run examples: examples/run_to_paper.py
  • Tests:

Highlighted Details

• Real, Pluggable Experiments: Drives PyTorch runs (CPU-fast, no download). New domains require only a YAML profile and engine pair.
• 8 Calibrated Agents: Typed JSON-schema I/O with validation and retries, operating under a shared audit trace.
• Literature-Grounded Hypotheses: Lit Scout pulls papers from arXiv/OpenAlex, ranks by citation impact, and distills techniques.
• Budget-Aware Funnel: Ideas progress through smoke → verify → full tiers, with cheap screening to eliminate bad ideas early.
• Engineered Loop: Features a parallel population funnel (propose N, smoke-screen all) and a self-stopping governor (halts on budget, round cap, or convergence).
• Self-Improving: Predicts experiment outcomes, scores against reality, and distills failures into a relevance-ranked, time-decaying skill library.
• Deterministic Harness: Prevents reward-hacking via two-phase frozen scoring, edit allowlists, and VerifiedRegistry number-grounding. Runs deterministically with MockLLM.

Maintenance & Community

The project is marked as a "research preview" with ongoing development. No specific community links (Discord, Slack) or notable contributors/sponsorships are detailed in the README. The CI badge indicates active automated testing.

Licensing & Compatibility

• License: MIT License.
• Compatibility: Permissive MIT license generally allows for commercial use and integration with closed-source projects.

Limitations & Caveats

The project is a "research preview," indicating it may be experimental or subject to significant changes. Further work is planned for container sandboxing to address residual boundaries. The system's own reviewer may deem results too marginal even if they outperform baselines, reflecting a strict scientific evaluation.