This project provides a C++ implementation of Stable Diffusion using the ncnn framework, targeting efficient inference on various platforms including x86 (Windows, Linux, macOS) and Android. It supports both text-to-image and image-to-image generation, aiming to offer a performant and resource-conscious alternative for users who need to run Stable Diffusion locally without heavy dependencies.

How It Works

The implementation leverages the ncnn framework, a high-performance neural network inference framework optimized for mobile and embedded devices. It breaks down the Stable Diffusion pipeline into its core components: CLIP for text embedding, a UNet model for iterative sampling, and an AutoencoderKL for latent space manipulation. The use of ncnn allows for optimized operations and dynamic shape support, enabling flexibility in resolution while maintaining efficiency.

Quick Start & Requirements

• x86 (Windows): Download pre-compiled binaries and models, place them in an assets folder, configure magic.txt, and run stable-diffusion.exe.
• x86 (Linux/macOS): Build ncnn, then build the demo using CMake. Download models to build/assets. Run ./stable-diffusion-ncnn.
• Android: Install the provided APK. Requires ~7GB RAM.
• Dependencies: C++ compiler (VS2019 for Windows), CMake, ncnn framework. Models (e.g., *-fp16.bin) are required.

Highlighted Details

• Supports both txt2img and img2img functionalities.
• Offers performance metrics for different hardware (i7-12700, Snapdragon865) and resolutions (512x512, 256x256).
• Features dynamic shape support for resolutions, requiring multiples of 128 and a minimum of 256.
• Includes options for "slow" (low RAM) and "fast" (high RAM) generation modes.

Maintenance & Community

The project appears to have recent updates (March 2023) adding img2img capabilities for Android and x86. Links to Zhihu and Bilibili are provided for more detailed explanations and demonstrations.

Licensing & Compatibility

The README does not explicitly state a license. It mentions compliance with the Stable Diffusion model agreement and requests notification for open-source projects using its ONNX models. Commercial use implications are not detailed.

Limitations & Caveats

The Android APK is noted to be slow and power-consuming, requiring 7GB of RAM. The project relies on external model weights ("Naifu") which are not included and must be sourced separately. The project statement requests users not to use it for illegal purposes.