Clicky is an open-source AI teacher companion for macOS, designed to integrate with the user's cursor, observe the screen, and communicate via voice and text. It offers developers a platform to customize, extend, or study its architecture, providing an interactive AI assistant directly on the desktop.

How It Works

This macOS menu bar application features a control panel and a transparent cursor overlay. It captures audio via push-to-talk, streams it to AssemblyAI for transcription, and sends transcripts with screenshots to Claude via SSE. Claude's responses are synthesized by ElevenLabs TTS. A unique feature allows Claude to embed [POINT:x,y:label:screenN] tags to direct the cursor across displays. All external API calls are proxied through a Cloudflare Worker for security.

Quick Start & Requirements

Setup is streamlined via Claude Code using a specific prompt, or manually. Manual setup requires macOS 14.2+, Xcode 15+, Node.js 18+, a Cloudflare account, and API keys for Anthropic, AssemblyAI, and ElevenLabs. Key steps include configuring the Cloudflare Worker (deploying or local dev), updating proxy URLs in the Swift code (leanring-buddy directory), and building via Xcode.

Highlighted Details

• Screen-aware AI desktop companion.
• Real-time transcription and AI response generation.
• AI-driven dynamic cursor control.
• Secure API key management via Cloudflare Worker.
• macOS menu bar integration with custom overlays.

Maintenance & Community

Contributions via Pull Requests are welcome. Feedback and inquiries can be directed to the maintainer on X (@farzatv). The project encourages leveraging Claude Code for easier development.

Licensing & Compatibility

The project's license is not explicitly stated, potentially impacting commercial adoption. Functionality is limited to macOS due to reliance on platform-specific frameworks like ScreenCaptureKit.

Limitations & Caveats

Requires macOS 14.2+. The project directory leanring-buddy contains a deliberate typo. The unspecified license necessitates clarification for integration. Setup involves managing multiple third-party API keys, which may incur costs.