This Python library and framework enables the creation of custom sign language translators, bridging communication gaps for the hearing-impaired. It supports translation between various sign languages and text, offering a flexible platform for researchers and developers to build region-specific solutions and contribute to sign language standardization.

How It Works

The project employs a modular approach for both sign-to-text and text-to-sign translation. For sign-to-text, it extracts features from videos using MediaPipe 3D landmarks, then transcribes and translates signs into multiple text languages. For text-to-sign, it offers a rule-based concatenation system that maps text to sign language video clips, and a deep learning approach using seq2seq models. The framework is designed to be extensible, allowing users to define custom language rules and fine-tune models with their own datasets.

Quick Start & Requirements

• Install: pip install sign-language-translator or pip install -e ".[all]" for full dependencies.
• Prerequisites: Python 3.7+, with optional dependencies like mediapipe for pose extraction.
• Documentation: https://sign-language-translator.readthedocs.io/
• Web Demo: https://huggingface.co/sltAI

Highlighted Details

• Supports full sentence translation, not just individual signs.
• Includes a rule-based system for generating synthetic training data.
• Provides pre-trained models for Pakistan Sign Language (23 hours of data) and supports English, Urdu, and Hindi text processing.
• Features a command-line interface (slt) for various tasks like translation, embedding, and asset management.

Maintenance & Community

The project is actively maintained, originating as a university project and continuing as a hobby project. Community contributions are encouraged for data collection, labeling, and code development. Resources for contribution are detailed in the README.

Licensing & Compatibility

Licensed under Apache 2.0. This license permits commercial and non-commercial use, modification, and incorporation into other works, provided appropriate credit is given to the original author.

Limitations & Caveats

While the project supports multiple languages and offers a framework for custom translators, the quality and coverage of translations are dependent on the availability and quality of training data for specific sign languages. The deep learning components for sign-to-text are marked as "COMING SOON."