NVIDIA NeMo is a comprehensive, cloud-native framework for developing and deploying generative AI models across Large Language Models (LLMs), Multimodal, Automatic Speech Recognition (ASR), and Text-to-Speech (TTS). It targets researchers and PyTorch developers, offering tools to efficiently create, customize, and scale AI models using pre-trained checkpoints and advanced distributed training techniques.

How It Works

NeMo 2.0 emphasizes modularity and ease of use, transitioning from YAML to Python-based configurations and adopting PyTorch Lightning's modular abstractions. It supports advanced distributed training strategies like Tensor Parallelism, Pipeline Parallelism, and Fully Sharded Data Parallelism (FSDP), leveraging NVIDIA Transformer Engine for FP8 training on Hopper GPUs and Megatron Core for scaling. This approach enables efficient training of massive models and facilitates experimentation with various components.

Quick Start & Requirements

• Installation: pip install "nemo_toolkit[all]" or via NGC containers.
• Prerequisites: Python 3.10+, PyTorch 2.5+, NVIDIA GPU (for training).
• Resources: NGC containers offer pre-built, optimized environments.
• Docs: NeMo Framework User Guide

Highlighted Details

• Supports training and fine-tuning of Hugging Face models via AutoModel.
• Offers advanced LLM alignment techniques (DPO, RLHF) and parameter-efficient fine-tuning (PEFT).
• Achieved near-linear performance scaling on 11,616 H100 GPUs for LLM pretraining in MLPerf Training v4.0.
• Includes NeMo Curator for accelerated video processing and Cosmos tokenizers for efficient visual data handling.

Maintenance & Community

• Actively developed by NVIDIA, with frequent updates and new features.
• Community contributions are welcomed. Discussions can be found on the NeMo Discussions board.

Licensing & Compatibility

• NeMo GitHub repository is licensed under Apache 2.0.
• Usage of NeMo containers is governed by the NVIDIA AI Product Agreement, which may have restrictions for commercial use.

Limitations & Caveats

• NeMo Framework Launcher is compatible only with NeMo version 1.0; NeMo-Run is recommended for NeMo 2.0.
• Windows support is currently not available.
• The NeMo Framework Launcher does not yet support ASR and TTS training.