NotaGen is a symbolic music generation model designed to produce high-quality classical sheet music by leveraging LLM training paradigms. It targets researchers and developers interested in advancing AI-driven music composition, offering a multi-stage training approach for enhanced musicality.

How It Works

NotaGen employs a three-stage training process: pre-training on a large corpus of musical pieces, fine-tuning on curated classical compositions with specific prompts (period, composer, instrumentation), and reinforcement learning using a novel CLaMP-DPO method. This RL approach optimizes musicality without requiring human annotations or predefined reward functions, aiming for more natural and sophisticated musical outputs.

Quick Start & Requirements

• Installation: Use Conda to create an environment (conda create --name notagen python=3.10), activate it (conda activate notagen), and install PyTorch with CUDA support (conda install pytorch==2.3.0 pytorch-cuda=11.8 -c pytorch -c nvidia). Then, install other dependencies (pip install accelerate optimum -r requirements.txt).
• Prerequisites: Python 3.10, PyTorch 2.3.0 with CUDA 11.8.
• Resources: NotaGen-large requires at least 24GB of GPU memory for training and inference.
• Demos: Online Gradio demo available on Huggingface Space, local Gradio demo, and a Colab notebook are provided. A ComfyUI node is also contributed.

Highlighted Details

• Offers pre-trained weights for small (110M), medium (244M), and large (516M) parameter models.
• Introduces CLaMP-DPO, a reinforcement learning method for improving musicality without human labels.
• NotaGen-X version includes post-training stages and refined fine-tuning for improved generation quality and instrument range.
• Supports data preprocessing for ABC notation to MusicXML conversion.

Maintenance & Community

• The project is associated with authors from multiple institutions, indicating potential academic backing.
• Links to a demo page and Huggingface Space are provided. Further instructions for adding new composer styles are available via issue #18.

Licensing & Compatibility

• The README does not explicitly state a license. The citation points to an arXiv preprint, suggesting it is a research project.

Limitations & Caveats

• Pre-trained weights are not usable for conditional generation based on period-composer-instrumentation.
• The README mentions a 2025 publication date for the cited paper, suggesting the project might be relatively new or still in active development.