Summary

ABC (Scalable Behavior Cloning with Open Data, Training, and Evaluation) offers an open framework for robotic manipulation tasks, enabling researchers and engineers to collect data, train, and evaluate behavior cloning policies. It aims to democratize advanced RL techniques.

How It Works

The project employs a Diffusion Transformer (ABC-DiT) architecture for behavior cloning, processing sequential states and camera inputs to predict actions. It emphasizes an end-to-end pipeline from data preparation to policy evaluation, leveraging large datasets and efficient training for complex manipulation skills.

Quick Start & Requirements

Installation requires uv and ffmpeg. A Python 3.12 virtual environment is managed via uv.

• Prerequisites: Python 3.12, uv, ffmpeg, ~35GB disk space for full data.
• DINOv3 Weights: Must be downloaded separately and placed in the cache directory, adhering to its license.
• Training: Default command uv run torchrun --standalone --nproc-per-node 8 train.py requires 8 GPUs; adjust for fewer GPUs/VRAM.
• Resources: Paper: https://arxiv.org/abs/2606.27375.

Highlighted Details

• Achieves ~0.048 training loss after 75k steps.
• Inference speed: ~2.6-3 iterations/sec on H100/H200.