This repository provides the official implementation for "Asynchronous Large Language Model Enhanced Planner for Autonomous Driving," targeting researchers and engineers in autonomous driving and AI planning. It offers an LLM-enhanced planning system that integrates with the NuPlan dataset and supports deployment on NVIDIA Jetson Orin devices via TensorRT optimization.

How It Works

The system asynchronously integrates a Large Language Model (LLM) with a real-time planner. This approach allows the LLM to provide high-level reasoning and planning guidance while the real-time planner handles immediate trajectory generation, aiming to improve planning efficiency and robustness. The architecture is designed for efficient inference, with specific optimizations for NVIDIA hardware.

Quick Start & Requirements

• Installation: Clone the repository, download NuPlan dataset, set environment variables (NUPLAN_MAPS_ROOT, NUPLAN_DATA_ROOT, NUPLAN_EXP_ROOT), and set up a Conda environment (Python 3.8 for Jetson, environment.yml for x86). Install dependencies via env_arm.sh (Jetson) or pip install -r requirements_asyncdriver.txt (x86).
• Prerequisites: NuPlan dataset, LLaMA-2-13b-chat-hf model, PDM checkpoint. NVIDIA Jetson Orin requires JetPack SDK.
• TensorRT/ONNX: Export LoRA model to ONNX, then convert to TensorRT engine. Modify inference scripts to specify the backend.
• Links: NuPlan Dataset

Highlighted Details

• Achieves significantly reduced inference latency on NVIDIA Jetson AGX Orin with TensorRT (FP16) at 0.1016s compared to PyTorch (0.3250s).
• Supports asynchronous inference with configurable intervals between LLM and real-time planner.
• Includes scripts for evaluation, training (GameFormer, Planning-QA, Reasoning1K), and model export to ONNX/TensorRT.
• Offers an orin_tensorrt branch for NVIDIA Jetson Orin deployment.

Maintenance & Community

The project is associated with AIR-THU and cites contributions from nuplan-devkit, GameFormer, tuplan_garage, and LLaMA. No specific community links (Discord/Slack) or roadmap are provided in the README.

Licensing & Compatibility

The repository's license is not explicitly stated in the README. It mentions being built upon other projects, which may have their own licensing terms. Compatibility for commercial use or closed-source linking is not detailed.

Limitations & Caveats

The README notes that LoRA fine-tuning support in JetPack 5.1.2 is limited, recommending ONNX export on an x86 host for Jetson deployment. A PDM checkpoint path needs to be updated in configuration files, though it's stated as not actively used in the current version. A specific line modification in simulator_llama4drive.py might be required if the planner is not found.