This repository explores the application of GPT-4V as an autonomous driving agent, providing a dataset of test images and detailed prompt/response logs for researchers and developers in the autonomous driving and AI fields. It aims to showcase GPT-4V's capabilities in understanding driving scenarios, reasoning, and decision-making.

How It Works

The project leverages GPT-4V's visual-language understanding to interpret driving scenes. It organizes test cases into categories like "Scenario Understanding," "Reasoning," and "Serving as a Driving Agent." Each case includes a .png image of a driving scenario and a .json file containing the specific prompts given to GPT-4V and its corresponding textual responses, demonstrating its decision-making process.

Quick Start & Requirements

• Access to GPT-4V API is required.
• No specific installation commands are provided; the repository serves as a data and results showcase.
• Users will need to interact with the GPT-4V API themselves using the provided data.

Highlighted Details

• Comprehensive dataset of driving scenarios with corresponding GPT-4V prompts and responses.
• Demonstrates GPT-4V's capabilities in weather understanding and handling corner cases.
• Includes examples of GPT-4V acting as a multi-task driving agent.
• Cites the technical report "On the Road with GPT-4V(ision): Early Explorations of Visual-Language Model on Autonomous Driving."

Maintenance & Community

• The repository is associated with PJLab-ADG.
• Contributions are welcomed via issues and pull requests.
• Links to other related projects from the team are provided.

Licensing & Compatibility

• Content is licensed under the MIT License.
• Permissive license allows for commercial use and integration with closed-source projects.

Limitations & Caveats

This repository primarily showcases results and data; it does not provide a runnable autonomous driving agent or code for direct integration. Users must have API access to GPT-4V and implement their own inference pipelines.