MinerU-Popo  by opendatalab

Universal OCR post-processing for structured documents

Created 2 months ago
261 stars

Top 97.2% on SourcePulse

GitHubView on GitHub
Project Summary

Summary

MinerU-Popo is a lightweight, universal framework enhancing OCR outputs by transforming page-level parsing into document-level semantic structures. It targets document analysis practitioners, providing crucial post-processing to unlock deeper understanding and improve downstream tasks.

How It Works

This project uses a 4B parameter model for four subtasks: table, text, title hierarchy, and image-text analysis. It tackles cross-page discontinuity and long document scalability via a Task-Oriented Data Engine, Dynamic Chunking/Synchronization for global consistency, and Document Enrichment for semantic tree construction and node splitting. This approach enables robust, scalable parsing beyond raw OCR.

Quick Start & Requirements

Install via source (pip install -r requirements.txt in Python 3.10 Conda env) or Docker (docker run -it --rm --gpus=all --ipc=host --network=host dockerrr8277/mineru-popo-vllm:latest). Download the model from Hugging Face (hf download DreamEternal/MinerU-Popo --local-dir models/Mineru-Popo). Configuration involves setting POPO_MODEL_PATH or editing inference function parameters. GPU acceleration is required for Docker.

Highlighted Details

  • Achieves a high Document Understanding Score (TEDS) of 90.6 after post-processing, significantly outperforming direct application of large foundation models (e.g., Qwen3-VL-32B at 78.0 TEDS).
  • Processes documents at a rate of 0.37 documents per second, demonstrating efficiency compared to larger, direct inference models.
  • Improves downstream retrieval and analysis accuracy on the ViDoRe V3 benchmark, with results like 84.4% accuracy in 'Content Structure' (C.S.) and 71.6% in 'Pharmaceutical' (Phar.), often matching or exceeding Visual RAG methods.

Maintenance & Community

No specific details regarding maintainers, community channels (e.g., Discord, Slack), or a public roadmap are provided in the README.

Licensing & Compatibility

The project is licensed under the permissive MIT License, allowing for broad use, modification, and distribution, including in commercial applications and closed-source projects.

Limitations & Caveats

The provided README does not explicitly detail known limitations, alpha/beta status, or specific unsupported platforms. The setup process involves multiple distinct steps (normalization, inference, tree building) which may require careful integration.

Health Check
Last Commit

1 month ago

Responsiveness

Inactive

Pull Requests (30d)
0
Issues (30d)
2
Star History
42 stars in the last 30 days

Explore Similar Projects

Starred by John Resig John Resig(Author of jQuery; Chief Software Architect at Khan Academy) and Elvis Saravia Elvis Saravia(Founder of DAIR.AI).

PageIndex by VectifyAI

0.0%
34k
Document index system for reasoning-based RAG
Created 1 year ago
Updated 4 days ago
Feedback? Help us improve.