This project provides a modular container build system for deploying the latest AI/ML packages on NVIDIA Jetson devices. It targets developers and researchers working with edge AI, offering pre-built and customizable Docker images for various machine learning frameworks, large language models, and related tools, simplifying complex dependency management for Jetson hardware.

How It Works

The system leverages Docker to create isolated environments for AI/ML workloads on Jetson. It defines packages as independent build recipes, allowing users to compose custom container images by selecting and combining desired ML frameworks, LLMs, and utilities. This approach simplifies dependency resolution and ensures compatibility with specific JetPack/L4T versions, abstracting away the complexities of manual compilation and environment setup.

Quick Start & Requirements

• Install via git clone https://github.com/dusty-nv/jetson-containers && cd jetson-containers && ./install.sh
• Requires Docker and NVIDIA Container Toolkit.
• Supports Ubuntu 22.04 (Python 3.10) and 24.04 (Python 3.12) with CUDA.
• Official docs: https://github.com/dusty-nv/jetson-containers
• Tutorials: https://github.com/dusty-nv/jetson-generative-ai-lab

Highlighted Details

• Supports a wide range of ML/LLM packages including PyTorch, TensorFlow, Hugging Face Transformers, Llama.cpp, Ollama, and Stable Diffusion.
• Offers automatic container image compatibility detection (autotag) for the installed JetPack/L4T version.
• Enables customization of CUDA, cuDNN, TensorRT, and Python versions during container builds.
• Includes support for SBSA (Arm Server Base System Architecture) devices like GH200/GB200.

Maintenance & Community

• Active community support via Discord.
• Regularly updated with new packages and JetPack versions.

Licensing & Compatibility

• Primarily uses MIT license for the build system. Individual packages may have their own licenses.
• Compatible with commercial use and closed-source applications.

Limitations & Caveats

The project focuses on NVIDIA Jetson platforms and may not be directly applicable to other edge devices or server-grade NVIDIA hardware without modification. Some advanced features or specific package versions might still be under active development or require manual tuning.