This library provides a Vercel AI SDK provider for Chrome's built-in Gemini Nano model, enabling developers to integrate on-device AI capabilities into their applications. It targets developers building Chrome extensions or web applications that can leverage local LLM processing for enhanced privacy and offline functionality.

How It Works

The chrome-ai package acts as an adapter, abstracting the underlying Gemini Nano model accessible via Chrome's experimental APIs. It integrates seamlessly with the Vercel AI SDK, allowing developers to use familiar functions like generateText, streamText, embedMany, generateObject, and streamObject with the local model. This approach simplifies the integration of on-device AI, offering a consistent interface regardless of the backend LLM.

Quick Start & Requirements

• Install via npm: npm install chrome-ai
• Requires Chrome version 127+ (Dev or Canary channel recommended, stable release expected July 17, 2024).
• Enable Chrome flags: chrome://flags/#prompt-api-for-gemini-nano and chrome://flags/#optimization-guide-on-device-model.
• Download the model via chrome://components/.
• Official documentation: Vercel AI SDK

Highlighted Details

• Supports text generation, streaming, and embeddings.
• Enables structured output generation (generateObject, streamObject) with Zod schemas.
• Offers a polyfill for broader browser compatibility (WebGPU/WebAssembly).
• Gemini Nano model runs locally, enhancing privacy and offline capabilities.

Maintenance & Community

• Project maintained by Jeason.
• Active development indicated by CI status and code coverage reports.
• Community engagement via Twitter: @jeasonstudio.

Licensing & Compatibility

• MIT License.
• Permissive license suitable for commercial use and integration into closed-source projects.

Limitations & Caveats

This library is under active development, with potential for errors and breaking changes. Chrome's Gemini Nano integration is experimental and subject to change. Tool/function calling is not yet supported, with efforts focused on prompt engineering solutions.