This NVIDIA AI Blueprint provides a framework for building customer service digital humans, leveraging NVIDIA's Tokkio workflow and ACE technologies. It targets development teams needing to integrate LLM or RAG applications with 3D animated avatars for more engaging customer interactions, offering faster resolutions than text-only AI.

How It Works

The blueprint utilizes NVIDIA NIM microservices for core functionalities: Audio2Face for avatar animation, Riva for speech AI, and ElevenLabs for speech synthesis. The avatar is rendered via Omniverse RTX or Unreal Engine. A separate RAG application generates responses, communicating with the digital human pipeline via REST API. This modular design allows customization of both the RAG component and the avatar.

Quick Start & Requirements

• Installation: Deployment involves Kubernetes/Helm charts and cloud provider resource provisioning. One-click scripts are available for digital avatar setup.
• Prerequisites: NVIDIA AI Enterprise or Developer License, NGC API key.
• Hardware: Minimum 2 NVIDIA GPUs (T4, A10, L4, L40S) for 1 stream, 4 GPUs for 3 streams. Requires 8-core CPU, 64GB RAM, 500GB disk space. Controller instance requires no GPU. RAG pipeline can run on CPU-only for initial familiarization or require 2x A100/L40s GPUs for self-hosted models.
• OS: Ubuntu 20.04 or 22.04.
• Links: NVIDIA API Catalog

Highlighted Details

• Offers three reference workflows: Tokkio LLM-RAG-ov (Omniverse), Tokkio LLM-RAG-ue (Unreal Engine Early Access), and Tokkio LLM-RAG-2D (2D Avatar Early Access).
• Supports 1, 3, or 6 stream options depending on the workflow.
• Designed for integration with LLM or RAG applications, with customization guidance provided.
• Highlights market trends: 80% of conversational offerings to embed generative AI by 2025, 75% of customer-facing applications to have emotion AI by 2030.

Maintenance & Community

• NVIDIA-backed project, leveraging NVIDIA NIM microservices.
• No specific community links (Discord/Slack) or roadmap details are provided in the README.

Licensing & Compatibility

• Requires NVIDIA AI Enterprise (NVAIE) License for self-hosting NIMs.
• Compatibility for commercial use depends on the NVAIE license terms.

Limitations & Caveats

• Unreal Engine and 2D Avatar workflows are in "Early Access."
• Requires significant technical expertise across DevOps, GenAI, and potentially animation/rendering and network engineering for deployment and customization.
• Network configuration for UDP/TCP ports is critical for production environments.