DiLu is a closed-loop, self-evolving framework for autonomous driving that integrates common-sense knowledge and memory with large language models (LLMs). It aims to provide a more intelligent and robust approach to autonomous driving challenges, targeting researchers and developers in the field.

How It Works

DiLu employs a four-module architecture: Environment, Reasoning, Reflection, and Memory. It leverages LLMs to process and reason about driving scenarios, incorporating a memory component for retaining and recalling past experiences. The reflection module allows for self-improvement by analyzing past actions and updating the memory, creating a knowledge-driven, evolving system.

Quick Start & Requirements

• Installation: Create a conda environment (conda create -n dilu python=3.8), activate it (conda activate dilu), and install dependencies (pip install -r requirements.txt).
• Prerequisites: Python 3.8, specific library versions (e.g., langchain==0.0.335, openai==0.28.1, chromadb==0.3.29), and OpenAI API keys (or Azure OpenAI credentials).
• Configuration: Set API keys and model names in config.yaml.
• Running: Execute python run_dilu.py. Simulation videos and logs are saved in the results folder.
• Visualization: Run python ./visualize_results.py and access the demo at http://127.0.0.1:7860.
• Documentation: Project page available here.

Highlighted Details

• Accepted by ICLR 2024.
• Features a knowledge-driven, self-evolving closed-loop system.
• Integrates LLMs with memory and reflection for enhanced reasoning.
• Supports both OpenAI and Azure OpenAI APIs.

Maintenance & Community

• Codes released January 2024. Paper available on Arxiv.
• Project page live since October 2023.

Licensing & Compatibility

• Released under the Apache 2.0 license.
• Permissive license suitable for commercial use and integration with closed-source projects.

Limitations & Caveats

The highway-env window may appear unresponsive during execution, but the terminal output indicates normal operation. Specific library versions are required, necessitating careful environment management.