This project implements an interactive planning agent for open-world multi-task scenarios, specifically within Minecraft. It targets researchers and developers working on embodied AI, LLM-driven agents, and reinforcement learning, offering a framework to enable agents to understand, plan, and execute complex tasks in dynamic environments.

How It Works

The agent utilizes a "Describe, Explain, Plan and Select" (DEPS) framework, leveraging Large Language Models (LLMs) to generate task plans. It interacts with a modified Minecraft simulator, receiving observations and using LLMs to break down high-level goals into actionable steps. The system is designed for interactive planning, allowing for human feedback or LLM-driven refinement of plans.

Quick Start & Requirements

• Install: Requires Python ≥ 3.9. Setup involves creating a Conda environment (conda create -n planner python=3.9), activating it, and installing dependencies via requirements.txt and a Git repository.
• Prerequisites: PyTorch (specifically torch==2.0.0.dev20230208+cu117 with CUDA 11.7), numpy, MineCLIP, and a modified MC-Simulator repository. OpenAI API keys are mandatory for LLM interaction.
• Setup: Requires cloning and installing MC-Simulator, preparing controller checkpoints, and configuring OpenAI API keys.
• Links: Arxiv Paper

Highlighted Details

• Implements the DEPS framework for LLM-driven interactive planning.
• Integrates with a modified Minecraft simulator for embodied AI research.
• Supports multiple OpenAI models (InstructGPT, Codex, ChatGPT) for planning.

Maintenance & Community

• The project notes a dependency on OpenAI's Codex, which was later cancelled, with plans to update to ChatGPT.
• No community links (Discord, Slack) or active development signals are present in the README.

Licensing & Compatibility

• The license is indicated as MIT via a shield, but the linked GitHub repository for MineCLIP shows a different license. Clarification is needed for commercial use.

Limitations & Caveats

The project's reliance on specific, potentially deprecated OpenAI APIs (Codex) and the need for a modified simulator fork indicate potential maintenance challenges and compatibility issues with current OpenAI offerings. The lack of explicit community support or recent updates raises concerns about its long-term viability.