Summary

This C/C++ library, transcribe.cpp, provides a high-performance speech-to-text inference engine leveraging the ggml runtime. It targets engineers and researchers needing efficient, on-device transcription capabilities across a wide array of hardware, including GPUs (Metal, Vulkan, CUDA) and CPUs. The project offers support for 16 distinct model families and over 60 variants, enabling fast, numerically verified transcription with streaming and batch processing options.

How It Works

The core of transcribe.cpp is its implementation on the ggml runtime, optimized for efficient tensor operations. It features multiple hardware acceleration backends: Metal for Apple Silicon, Vulkan for cross-platform GPU support (Linux/Windows), and CUDA for NVIDIA GPUs. A tinyBLAS-accelerated CPU path provides a fallback for systems without dedicated GPUs. This multi-backend approach ensures broad compatibility and high inference speeds, with models being numerically verified and Word Error Rate (WER) tested against reference implementations.

Quick Start & Requirements

Compilation is managed via CMake. A basic build involves cmake -B build && cmake --build build. For GPU acceleration, specific CMake flags are required: -DTRANSCRIBE_VULKAN=ON (requires libvulkan-dev, glslc, libopenblas-dev on Debian/Ubuntu) or -DTRANSCRIBE_CUDA=ON (requires CUDA toolkit). Metal support is automatic on Apple Silicon. libopenblas-dev is recommended for significant CPU performance gains. Pre-built GGUF models are available on Hugging Face.

Highlighted Details

• Supports 16 model families and 60+ variants, including Whisper, Parakeet, Canary, and Granite Speech.
• All published handy-computer models are numerically verified and WER-tested.
• Offers fast GPU inference via Metal, Vulkan, and CUDA backends.
• Provides official bindings for Python, TypeScript/JavaScript, Rust, and Swift.

Maintenance & Community

The project is supported by Mozilla AI & BiR Program (research funding), Hugging Face (model hosting), Modal (GPU credits for validation), and Blacksmith (CI runners). No direct community links (e.g., Discord, Slack) are listed in the README.

Licensing & Compatibility

transcribe.cpp is released under the MIT license. Vendored third-party components, including ggml and miniz, are also MIT-licensed, permitting broad compatibility for commercial use and integration into closed-source projects.

Limitations & Caveats

Some specialized models, such as MedASR, may be gated. Building with specific GPU backends necessitates the installation of corresponding SDKs and drivers. The project focuses on inference, not model training.