This repository provides an upgraded OCR solution for converting mathematical formula images into LaTeX expressions. It's designed for researchers and developers working on mathematical content digitization, offering a robust pipeline for accurate LaTeX generation from visual inputs.

How It Works

The core approach utilizes a CNN for feature extraction, followed by a bidirectional RNN on the height dimension to capture contextual dependencies in the image. A GRU-based decoder with an attention mechanism then generates the LaTeX output. Positional encoding, inspired by Transformers, is incorporated into the CNN's final layer. The system handles data preprocessing, including tokenization, padding, and bucketing, to optimize training.

Quick Start & Requirements

• Install: make im2katex-inference for testing.
• Prerequisites: ImageMagick (Linux: sudo apt install imagemagick, Mac: brew install imagemagick), pdflatex.
• Demo: Web-based demo available via make server.
• Docs: https://guillaumegenthial.github.io/image-to-latex.html

Highlighted Details

• Implements three models: im2katex (image to LaTeX), errorchecker (syntax correction, currently deprecated), and dismodel (discriminator for improving LaTeX renderability).
• Supports training on handwritten, printed, or merged datasets.
• Offers both greedy and beam search decoding for LaTeX generation.
• Pre-trained weights are available via BaiduDisk.

Maintenance & Community

• The project references OpenAI's research and provides links to related GitHub repositories and explanations.
• No explicit community links (Discord/Slack) or roadmap are provided in the README.

Licensing & Compatibility

• The README does not explicitly state a license. The presence of links to OpenAI's "Requests For Research" and other repositories suggests potential non-commercial or research-focused usage, but explicit licensing terms are absent.

Limitations & Caveats

The errorchecker model, intended for correcting LaTeX syntax errors, is noted as deprecated due to poor performance. The dismodel is an ongoing optimization effort. The lack of explicit licensing information may pose a barrier to commercial adoption.