This repository provides PyTorch code for ViTSTR, a Vision Transformer-based model for fast and efficient scene text recognition. It offers comparable accuracy to state-of-the-art models with significantly fewer parameters and FLOPS, making it suitable for researchers and developers working on OCR and text detection tasks.

How It Works

ViTSTR leverages a pre-trained Vision Transformer (ViT) architecture for scene text recognition. This approach capitalizes on the parallel processing capabilities inherent in ViTs, leading to faster inference times compared to traditional recurrent or convolutional models. The model is designed as a single-stage architecture, simplifying its implementation and training.

Quick Start & Requirements

• Install: pip3 install -r requirements.txt
• Prerequisites: Python 3, PyTorch. GPU recommended for training and faster inference.
• Inference: python3 infer.py --image <path_to_image> --model <model_url_or_path>
• Demo: https://github.com/roatienza/deep-text-recognition-benchmark/blob/main/demo_image/demo_1.png
• Model Weights: Available for Tiny, Small, and Base configurations, with and without augmentation.

Highlighted Details

• Achieves competitive accuracy on various benchmarks (IIIT, SVT, IC03, IC15, etc.) with fewer parameters and FLOPS.
• Demonstrates fast inference times: ~2.57ms on Quadro RTX 6000, ~28ms on CPU.
• Offers quantized models optimized for x86 and Raspberry Pi 4.
• Supports training from scratch with detailed configurations for data augmentation and multi-GPU setups.

Maintenance & Community

• The project is based on a fork of CLOVA AI Deep Text Recognition Benchmark.
• The primary contributor is Rowel Atienza.
• The paper associated with this work is "Vision Transformer for Fast and Efficient Scene Text Recognition" (ICDAR 2021).

Licensing & Compatibility

• The repository does not explicitly state a license in the README. However, being a fork of another project, its licensing status may depend on the original repository. Users should verify licensing for commercial use.

Limitations & Caveats

• The README does not specify the exact license, which may pose a risk for commercial adoption.
• Training requires significant computational resources, especially for larger models and extensive data augmentation.