This repository provides the official codebase for "Let's Think Outside the Box: Exploring Leap-of-Thought in Large Language Models with Creative Humor Generation," a CVPR 2024 paper. It explores "Leap-of-Thought" (LoT), a non-sequential thinking skill in multimodal LLMs, using the Oogiri humor generation game as a testbed. The project targets researchers and developers interested in advancing LLM creativity and multimodal reasoning.

How It Works

The project investigates LoT by challenging LLMs to generate creative and humorous responses to multimodal inputs within the Oogiri game framework. This approach aims to push LLMs beyond sequential reasoning, similar to Chain-of-Thought, by requiring them to "think outside the box" for unexpected and witty outputs.

Quick Start & Requirements

• Install dependencies: pip install -r requirements.txt
• Run inference: python inference.py
• Launch Gradio demo: python gradio_demo.py
• Requirements include Python packages listed in requirements.txt. Checkpoints and datasets are available on ModelScope and Hugging Face.

Highlighted Details

• Explores "Leap-of-Thought" (LoT) in multimodal LLMs.
• Utilizes the Oogiri game for creative humor generation.
• Addresses multimodal inputs: Image-to-Text (I2T), Text-to-Text (T2T), and Image-to-Text (IT2T).
• Official codebase for a CVPR 2024 accepted paper.

Maintenance & Community

The project has released its dataset and checkpoints. Links to the project page and paper are provided.

Licensing & Compatibility

The repository does not explicitly state a license. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

No specific limitations or caveats are mentioned in the provided README. The project appears to be focused on research and demonstration.