This repository provides the Factorio Learning Environment (FLE), a framework for developing and evaluating Large Language Model (LLM) agents within the game Factorio. It offers two modes: "Lab-play" for structured tasks and "Open-play" for unbounded factory building, aiming to assess LLMs' spatial reasoning and complex automation capabilities.

How It Works

FLE enables LLM agents to interact with Factorio through a code synthesis REPL pattern. Agents observe the game state via program output streams, generate Python code for actions using a provided API, and receive feedback. This feedback loop, coupled with a namespace for state management and typed exceptions for debugging, allows agents to build complex representations and self-correct. The environment supports custom tools, implemented in Python and Lua, for agent interaction.

Quick Start & Requirements

• Install: pip install -e .
• Prerequisites: Factorio (version 1.1.110), Python 3.10+, Docker.
• Setup: Requires purchasing Factorio, downgrading to version 1.1.110, configuring Docker, and setting up a database (Postgres or SQLite). Firewall configuration may be needed for remote servers.
• Links: Leaderboard, Paper, Website

Highlighted Details

• Supports two main modes: Lab-play (24 structured tasks) and Open-play (unbounded factory building).
• Includes a visual agent that takes map renderings as input.
• Offers a comprehensive API with ~27 agent tools for game interaction.
• Benchmarks show significant performance differences between headless and client-based Factorio server execution, with headless offering higher throughput.
• The interpreter overhead for executing Python policies is substantial, reducing ops/sec by up to 69%.

Maintenance & Community

• The project is actively maintained, with recent updates including the visual agent.
• Community channels are not explicitly mentioned in the README.

Licensing & Compatibility

• The repository's license is not explicitly stated in the provided README text.

Limitations & Caveats

• Requires a specific, older version of Factorio (1.1.110), necessitating a downgrade.
• The setup process involves multiple steps including game installation, Docker configuration, and database setup, which can be time-consuming.
• Performance benchmarks indicate that the Python interpreter layer significantly impacts execution speed.