Orca is a general world foundation model centered on Next-State-Prediction, aiming to understand, predict, and act upon the world. It learns a unified latent space from multimodal signals (visual, language), targeting AI researchers and practitioners. Its core benefit is enhanced world modeling and downstream performance via unified state-transition modeling.

How It Works

Orca employs an Encoder-Decoder architecture with Next-State-Prediction as its core objective, modeling world-state transitions through implicit dynamics and explicit conditions. It integrates "unconscious" learning from continuous video with "conscious" learning from language events and VQA. Post-pre-training, the encoder is frozen, and lightweight, modality-specific decoders (text, image, action) are trained for readout, enabling efficient adaptation and strong performance.

Quick Start & Requirements

Clone the repo and install evaluation dependencies:

git clone https://github.com/orca-wm/Orca.git
cd Orca/evaluation
python -m pip install -r requirements-data.txt

Download evaluation datasets (e.g., PRICE, SWITCH) using provided scripts. The BAAI/Orca-4B checkpoint is available on Hugging Face. Project page and technical report links offer further details. Setup requires Python; significant compute resources are implied.

Highlighted Details

• Text Generation: Orca-4B shows competitive results on TemporalBench, MVBench, SWITCH, and 3DSRBench, outperforming models like Qwen3.5.
• Image Prediction: Orca-4B+2 achieves strong performance on PRICE-V0.1 (avg. 59.8±10.9), surpassing Gemini 3.1 Pro and GPT 5.4.
• Action Generation: Orca demonstrates robust real-robot manipulation performance, with higher success rates and robustness scores than V-JEPA and Qwen3.5.
• Scaling: Downstream readout performance improves with pre-training scale, validating the unified latent space and frozen-backbone readout approach.

Maintenance & Community

Recent releases include the Orca-4B checkpoint (July 14, 2026) and technical report (June 29, 2026). Planned releases include Orca-0.8B and fine-tuning code. Community support is primarily via WeChat contact.

Licensing & Compatibility

The README does not specify a software license, hindering assessment of its terms for use, modification, and distribution, especially for commercial applications.

Limitations & Caveats

The project is under active development; Orca-0.8B and fine-tuning code are upcoming. The current release focuses on the Orca-4B checkpoint and evaluation code. The lack of a clear license is a significant adoption blocker.