This library provides a JavaScript implementation of diffusion models, enabling users to run them directly in Node.js or the browser. It targets developers and researchers looking to integrate or experiment with generative AI capabilities client-side or on serverless Node.js environments.

How It Works

The project leverages ONNX Runtime for efficient model execution, with custom builds tailored for WebAssembly and Node.js environments. It supports GPU acceleration via CUDA, DirectML, and WebGPU, aiming to provide near-native performance for diffusion models in JavaScript.

Quick Start & Requirements

• Install: npm i @aislamov/diffusers.js
• Node.js GPU Support: Requires a custom build of onnxruntime-node with merged DML and CUDA support (as of README, not yet released).
• Browser: Requires WebGPU support.
• Model Loading: Models optimized for GPU will fail without appropriate backend support; use revision: 'cpu' for CPU execution.
• Examples: Available for React and Node.js.

Highlighted Details

• ONNX Runtime for WebAssembly and Node.js.
• GPU acceleration via CUDA, DirectML, WebGPU.
• Custom ONNX Runtime builds and Emscripten fixes for compatibility.

Maintenance & Community

The primary contributor is dakenf, who has made significant contributions to ONNX Runtime and Emscripten. The project aims to merge its fixes into official releases.

Licensing & Compatibility

The license is not explicitly stated in the README. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

Node.js GPU support relies on unreleased ONNX Runtime features. Browser execution is dependent on WebGPU availability. The project involves custom builds and fixes for underlying dependencies, indicating potential maintenance overhead.