Automated Weak-to-Strong Research provides a sandbox environment for exploring AI alignment strategies, specifically addressing the challenge of weak-to-strong generalization. It targets AI safety researchers and engineers seeking to develop methods for aligning AI systems that may surpass human evaluation capabilities. The project offers datasets, baseline implementations, and an automated researcher agent to facilitate systematic experimentation and accelerate progress in AI safety.

How It Works

The core methodology revolves around weak-to-strong generalization, where a weaker AI model is trained on labeled data, then used to pseudo-label a larger unlabeled dataset. A stronger AI model is subsequently trained on this pseudo-labeled data. Performance Gap Recovery (PGR), calculated as (transfer_acc - weak_acc) / (strong_acc - weak_acc), quantifies how effectively the strong model recovers ground truth performance. The project features an automated researcher, powered by Claude, which iteratively proposes, implements, and evaluates novel alignment ideas within this framework, leveraging a dedicated evaluation API to prevent data leakage.

Quick Start & Requirements

Installation involves synchronizing dependencies using uv sync. Datasets (chat, math, code) require unpacking and preparation via tar xzf labeled_data.tar.gz and python scripts/prepare_data.py. Baselines can be run using python run.py --idea <idea_name> .... A Flask-based dashboard server must be started with python run.py server --port 8000. Key requirements include PyTorch, Transformers, Unsloth, vLLM, Anthropic SDK, Flask, boto3, and RunPod. An ANTHROPIC_API_KEY is mandatory for all modes. Local Docker mode requires building an image (./scripts/docker-build-push.sh). RunPod mode necessitates RUNPOD_API_KEY, RUNPOD_TEMPLATE_ID, S3 credentials (S3_BUCKET, S3_ENDPOINT_URL, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY), and optionally WANDB_API_KEY.

Highlighted Details

• Features a Claude-powered automated researcher agent that autonomously proposes, implements, and evaluates new alignment ideas.
• Includes a web-based dashboard for experiment management, monitoring, an evaluation API (keeping ground truth server-side), and a comparative leaderboard.
• Provides implementations for several baseline ideas, including vanilla_w2s, train_only_on_confident_labels, critic, ue_zeroshot, and ue_fewshot.
• Supports multiple deployment modes: local subprocess, local Docker container with GPU passthrough, and distributed cloud execution on RunPod.

Maintenance & Community

No specific information regarding maintainers, community channels (like Discord/Slack), sponsorships, or roadmaps is present in the provided README.

Licensing & Compatibility

The project is released under the MIT license, which is permissive and generally compatible with commercial use and closed-source linking.

Limitations & Caveats

The "Local (subprocess)" execution mode may not provide sufficient isolation, potentially allowing the automated researcher to access ground truth labels, compromising result validity. The ue_fewshot baseline variant is noted as not being used in main experiments due to limitations with the Qwen3-4B-Base model's few-shot capabilities. Deployment, especially to RunPod, requires significant setup involving API keys and cloud infrastructure configuration.