Qinglong Captions is an AI-powered media processing and document translation toolkit designed for technical users. It streamlines complex workflows like video/image captioning, OCR, tagging, and audio separation, built upon the efficient Lance dataset format, offering a unified platform for diverse AI media tasks.

How It Works

The toolkit employs a GUI-driven pipeline (Import -> Split -> Tagger -> Caption -> Export) utilizing Lance datasets for intermediate storage. It supports flexible captioning backends, including cloud APIs, OpenAI-compatible servers, and local VLM/ALM/OCR models. Novelty lies in its integration of advanced features like audio stem separation to MIDI and Image2PSD workflows within a cohesive framework.

Quick Start & Requirements

• Installation:
  • Windows: powershell .\1.install-uv-qinglong.ps1
  • Linux: chmod +x "./0.install pwsh.sh"; sudo bash "./0.install pwsh.sh"; pwsh ./1.install-uv-qinglong.ps1
• Prerequisites: Windows/Linux (Linux installer x86_64 only; ARM64 requires manual pwsh 7+ install). Python >=3.10,<3.13. uv package manager (installer attempts to install). PowerShell 5.1+ (Windows) or pwsh (Linux). GPU optional for base workflow, but recommended/required for local VLM/OCR/translation and Image2PSD, demanding significant GPU memory and disk space. Models are downloaded on first use.
• Links: GUI manual, configuration guide, and troubleshooting guide are detailed within the README.

Highlighted Details

• Audio Separation: Advanced stem separation to MIDI via MuScriptor, with instrument family mapping and flexible preview options (MIDI, original/synthesized comparison). Supports SOCKS proxy for SoundFont downloads.
• Image2PSD: Integrates the see-through model for single-image layer decomposition.
• Multimodal Captioning: Supports local OCR, VLM, and ALM models alongside cloud providers, configurable via API keys and base URLs.
• Lance Datasets: Central to the workflow, enabling efficient data handling across splitting, tagging, captioning, and export stages.

Maintenance & Community

No specific community links (Discord, Slack) or prominent contributors/sponsorships are detailed in the README. A changelog is available for release history.

Licensing & Compatibility

The project uses a root LICENSE file (type unspecified). Third-party components and downloaded models are subject to their own licenses and access terms, requiring careful review for commercial use or integration into closed-source projects.

Limitations & Caveats

The Linux installer is x86_64 specific; ARM64 requires manual pwsh setup. API keys might be exposed in command-line arguments for some captioning paths, necessitating log redaction. The --cloud GUI option lacks built-in authentication and should not be exposed publicly. Gated Hugging Face models require explicit term acceptance and HF_TOKEN.