This project provides a hacky integration for Meta Rayban Smart Glasses, enabling users to send photos to GPT-4 Vision via voice commands and Messenger. It targets Meta Glasses users and developers looking to experiment with AI-powered interactions on the hardware.

How It Works

The system comprises a Node.js (bun) backend server that exposes a REST API endpoint for GPT-4 Vision integration. A browser bookmarklet, injected into messenger.com, observes for new image messages. Upon detecting an image, it extracts the image URL and forwards it to the backend API. The backend then processes the image with GPT-4 Vision and logs the interaction.

Quick Start & Requirements

• Install/Run: bun install followed by bun run dev for the server.
• Prerequisites: Meta Rayban Smart Glasses, OpenAI API Key, alternative Facebook/Messenger account, bun runtime.
• Setup: Requires creating a .env file with API keys and injecting a bookmarklet into messenger.com.
• Docs: Bookmarklet Code Breakdown

Highlighted Details

• Leverages GPT-4 Vision for image analysis via Meta Glasses.
• Utilizes a bookmarklet for Messenger integration, a technique noted as "hacky" and prone to breaking.
• Demonstrates a proof-of-concept for voice-activated AI interactions with smart glasses.

Maintenance & Community

• Developed by Devon Crebbin.
• No explicit community channels or roadmap mentioned.

Licensing & Compatibility

• No license specified in the README.
• Compatibility for commercial use or closed-source linking is not addressed.

Limitations & Caveats

The bookmarklet relies on obfuscated Messenger CSS class names, making it highly susceptible to breaking with future Messenger updates. The use of DOMNodeInserted is flagged as bad practice and deprecated. The project is presented as a "hacky way" and a proof-of-concept, awaiting official SDK support for robust integration.