XR Blocks is a lightweight, cross-platform JavaScript library designed for rapid prototyping of advanced XR and AI experiences. It targets developers looking to build immersive applications that integrate features like hand tracking, gesture recognition, and AI-powered world understanding, offering a unified SDK for both XR devices and desktop browsers.

How It Works

Built on top of three.js, XR Blocks provides a developer-friendly SDK that simplifies complex XR and AI integrations. It enables advanced hand tracking and custom gesture recognition using on-device models (TensorFlow Lite/PyTorch), alongside world understanding capabilities like depth sensing and object recognition powered by Gemini. The framework's core advantage lies in its cross-platform compatibility, allowing developers to write code once and deploy to WebXR-compatible devices and desktop Chrome browsers, streamlining the prototyping process.

Quick Start & Requirements

• Installation: Clone the repository (git clone --depth=1 git@github.com:google/xrblocks.git), navigate into the directory (cd xrblocks), and install dependencies with npm ci.
• Running Demos: Serve samples and demos locally via http://localhost:8080/ using npm run serve.
• Development: For active SDK development, use npm run dev to build in watch mode and serve locally.
• Prerequisites: Node.js and npm are required. The library targets Chrome v136+ with WebXR support on XR devices (e.g., Galaxy XR) and desktop Chrome browsers.
• Documentation Links: Links for Manual, Templates, Demos, Gem, YouTube, arXiv, and Blog are mentioned but not provided in the README.

Highlighted Details

• Advanced hand tracking and custom gesture recognition with options for gesture detection (pinch, fist, point, etc.).
• World understanding features including depth sensing, geometry-aware physics, and object recognition.
• Seamless integration with Gemini for multimodal understanding and live conversational AI experiences.
• Cross-platform deployment to WebXR-enabled devices and desktop Chrome browsers.

Maintenance & Community

XR Blocks is actively maintained by the XR Labs team and external collaborators, fostering an AI + XR community. While not an officially supported Google product, it is presented as a visionary roadmap through blog posts and white papers.

Licensing & Compatibility

The README does not explicitly state a license. It is an open-source SDK. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

This project is not an officially supported Google product and is not eligible for the Google Open Source Software Vulnerability Rewards Program. Certain features require explicit user permissions, and denied permissions may impact functionality. Secure handling of API keys for AI features (Gemini) is critical, as they should never be committed to source control or exposed client-side.