This proposal outlines a JavaScript API for web applications to interact with language models provided by the browser or operating system. It aims to offer web developers a standardized way to leverage LLM capabilities locally, enhancing privacy, performance, and offline functionality compared to cloud-based APIs.

How It Works

The API provides a session-based model where developers create a LanguageModel instance, optionally configuring it with system prompts and expected input types (text, image, audio). Interactions are performed via prompt() for single responses or promptStreaming() for token-by-token output. The API abstracts tokenization and model-specific details, allowing developers to focus on prompt engineering. It supports multimodal inputs, structured output constraints (JSON schema, RegExp), and session cloning for managing conversational history.

Quick Start & Requirements

• Install/Run: JavaScript API, intended for browser environments.
• Prerequisites: A browser implementing the API and a language model (on-device or cloud-based).
• Links: Explainer, Web IDL

Highlighted Details

• Supports text, image, and audio inputs, with potential for future multimodal extensions.
• Enables structured output via JSON schema or RegExp constraints for programmatic response handling.
• Offers session management features like cloning and explicit destruction for memory control.
• Includes availability() and monitor() for managing downloads and user experience.

Maintenance & Community

This is an early design sketch by the Chrome built-in AI team, seeking feedback. Browser engine support is currently limited to prototyping in Chromium.

Licensing & Compatibility

The repository itself is not licensed. The API proposal is a web standard effort.

Limitations & Caveats

This is an experimental proposal, not yet approved for shipping in Chrome. Browser implementation and language model quality are not guaranteed and vary by implementation. The API does not specify output languages or guarantee on-device execution.