Summary

TurboMeta provides a Chinese-language multimodal AI assistant for Ray-Ban Meta smart glasses, enabling real-time conversations, image recognition, and live streaming to enhance user experience.

How It Works

This project integrates Ray-Ban Meta's DAT SDK with Aliyun's Tongyi Qianwen multimodal AI models (Omni-Realtime, VL-Plus, TTS-Flash). It innovatively uses Siri Shortcuts and App Intents for background camera access and AI interaction, overcoming Meta's SDK limitations. The system supports both iOS and Android platforms, offering a seamless bridge between the glasses' hardware and powerful cloud AI.

Quick Start & Requirements

• Primary Install: Compile from source (Xcode) or sideload IPA for iOS; install APK for Android. Download releases from GitHub Releases.
• Prerequisites: Ray-Ban Meta smart glasses (firmware v20+), iPhone (iOS 17.0+) or Android (8.0+), stable internet, Meta AI App with Developer Mode enabled, and Aliyun API keys for Tongyi Qianwen models.
• Setup: Requires obtaining and configuring Aliyun API keys within the app. Estimated setup time depends on user familiarity with sideloading and API key acquisition.

Highlighted Details

• Multimodal AI: Features "Live AI" modes (museum guide, blind assist, reading, translation) and "Quick Vision" (object/scene recognition, health analysis).
• Siri & Action Button Integration: Enables voice-activated AI features via Siri Shortcuts or the iPhone 15 Pro Action Button, allowing hands-free operation.
• RTMP Live Streaming: Supports streaming directly from the glasses to platforms like YouTube, Twitch, Bilibili, and TikTok.
• OpenRouter Support: Allows integration with over 500 AI models, including GPT-5, Claude 4.5, and Gemini 3, offering flexibility in AI backend choice.

Maintenance & Community

The project shows recent activity with version 1.5.0 introducing significant features. Community support is indicated via a "Buy Me a Coffee" link, suggesting ongoing developer engagement.

Licensing & Compatibility

The project is licensed under MIT, but parts are based on Meta's original DAT SDK code, potentially subject to Meta's licensing terms. Commercial use requires adherence to both Meta's SDK terms and Aliyun's API service agreements. Compatibility is limited to specific Ray-Ban Meta hardware and supported mobile OS versions.

Limitations & Caveats

Enabling Developer Mode on the Ray-Ban Meta glasses is a mandatory, non-trivial setup step. Users must acquire and manage their own Aliyun API keys, which may incur costs. iOS installation via sideloading requires specific tools and periodic re-signing for free developer accounts. The application is hardware-specific to Ray-Ban Meta glasses.