This Unity plugin enables developers to integrate Google's MediaPipe framework directly into their Unity projects, allowing for the execution of MediaPipe's advanced perception pipelines within a game development environment. It targets Unity developers seeking to leverage real-time computer vision and machine learning capabilities, such as hand tracking, face detection, and pose estimation, directly within their interactive applications.

How It Works

The plugin provides C# bindings for MediaPipe's C++ API, facilitating the porting of MediaPipe's functionality to C#. This approach allows developers to write and run MediaPipe graphs, including custom calculators, directly from Unity scripts. While this offers flexibility, it may introduce performance overhead for frequent API calls compared to a pure C++ implementation.

Quick Start & Requirements

• Installation: Download pre-built packages (.unitypackage or .zip) from the releases page.
• Prerequisites: Unity (>= 2022.3). For Android builds, libc++_shared.so must be included in the APK.
• Setup: Pre-built packages simplify setup; building from source requires following a build guide.
• Resources: Releases Page

Highlighted Details

• Supports a wide range of MediaPipe solutions including object detection, image segmentation, hand/face/pose landmark detection, and audio classification across multiple platforms.
• Offers flexibility to run official MediaPipe solutions, custom calculators, and CalculatorGraphs within Unity.
• Provides pre-built packages for easier integration, with options for full source code or stripped binaries.
• Includes sample scenes to demonstrate various MediaPipe solutions within Unity.

Maintenance & Community

• The project is actively maintained by homuler.
• Community support channels are not explicitly mentioned in the README.

Licensing & Compatibility

• License: MIT. Note that MediaPipe itself is under Apache License 2.0, and other components may have different licenses (e.g., emscripten, FontAwesome).
• Compatibility: Generally compatible with commercial and closed-source projects due to the MIT license, but users should verify third-party licenses.

Limitations & Caveats

• GPU inference is not supported on macOS and Windows.
• Native library crashes can cause Unity Editor or application instability, with limited SIGABRT handling on Windows.
• For GPU inference, the OpenGL Core API is not supported; Vulkan is recommended on PC standalone builds.
• Running MediaPipe on Windows is noted as experimental.