This project provides AI-powered audio effects, generators, and analyzers for Audacity, enabling users to perform tasks like music separation, noise suppression, and audio transcription entirely offline. It targets Audacity users seeking to integrate advanced AI capabilities directly into their audio editing workflow without requiring internet connectivity.

How It Works

The plugins leverage Intel's OpenVINO toolkit to run various AI models efficiently on local hardware, including CPUs, GPUs, and NPUs. This approach allows for high-performance inference of complex models like Meta's MusicGen for music generation, Demucs v4 for music separation, and whisper.cpp for transcription, all optimized for local execution.

Quick Start & Requirements

• Installation packages and instructions for Windows are available at the project's GitHub releases page.
• Build instructions for Windows and Linux are provided.
• Requires Audacity. Specific hardware accelerators (CPU, GPU, NPU) are leveraged by OpenVINO.

Highlighted Details

• Music Separation: Splits audio into stems (Drums, Bass, Vocals, Other) using Meta's Demucs v4.
• Music Generation: Utilizes Meta's MusicGen (Small and Small-Stereo variants) for creating or continuing music snippets.
• Transcription: Employs whisper.cpp for generating transcriptions or translations of spoken audio.
• Noise Suppression: Implements models like noise-suppression-denseunet-ll and DeepFilterNet2/3 for background noise removal.

Maintenance & Community

• Contributions are welcomed via pull requests.
• Issues for questions, bug reports, feature requests, and feedback can be submitted on the GitHub repository.
• Acknowledgements are given to the Audacity development team & Muse Group.

Licensing & Compatibility

• The project utilizes various open-source models and libraries, including those under permissive licenses. Specific licensing details for the plugins themselves are not explicitly stated in the README, but dependencies like Audacity and whisper.cpp have their own licenses.
• The use of OpenVINO™ implies compatibility with Intel hardware accelerators.

Limitations & Caveats

• The README primarily details Windows installation, with Linux build instructions also provided, suggesting potential platform-specific nuances.
• The project relies on specific AI models which may have varying performance characteristics and resource requirements.