Code World Model (CWM) is a 32-billion-parameter open-weights Large Language Model (LLM) designed to advance research in code generation and reasoning using world models. It targets AI researchers and engineers working on code-related AI tasks, offering a foundation for developing more capable code generation and understanding systems. The primary benefit is enabling deeper research into how LLMs can represent and reason about program state and execution.

How It Works

CWM employs a two-stage training process. It is first mid-trained on a large corpus of observation-action trajectories derived from Python execution traces and agentic interactions within containerized environments. This is followed by extensive multi-task reinforcement learning (RL) post-training, focusing on verifiable coding, mathematical problem-solving, and multi-turn software engineering scenarios. This approach aims to imbue the model with a better understanding of code's effect on system state.

Quick Start & Requirements

Environment setup is managed via micromamba env create -f environment.yaml -n CWM. Running evaluations and demos requires significant hardware: 160GB of combined GPU VRAM (e.g., two Nvidia H100 GPUs) and RDMA (Mellanox 5 InfiniBand or AWS EFA). Model weights are available on Hugging Face and as PyTorch checkpoints (DCP format) requiring license acceptance. Links to the accompanying Code World Model Tech Report and Model Card are provided for further details.

Highlighted Details

• Reproduces reported numbers on key benchmarks including SWE-bench Verified, LiveCodeBench, AIME, and MATH.
• Provides tools for local inference using Hugging Face weights or PyTorch DCP weights via a Fastgen server.
• Includes demos showcasing capabilities, such as using CWM as a neural debugger.
• Requires a specific system prompt for optimal inference performance.

Maintenance & Community

The project is associated with the FAIR CodeGen Team at Meta. Specific community channels (e.g., Discord, Slack), detailed contributor information, or a public roadmap are not explicitly detailed in the README.

Licensing & Compatibility

The code within this repository is released under a BSD-3 license. However, the model weights are distributed under a custom license, accessible via a dedicated CWM License page. Users must review this custom license for details regarding commercial use or integration into closed-source projects.

Limitations & Caveats

The substantial hardware requirements (160GB VRAM, RDMA) present a significant barrier to entry for many researchers and developers. The model weights are subject to a custom license, which may impose restrictions on usage beyond those typical for open-source code. Access to weights requires a request and approval process, which historically could take up to an hour.