SkyPaint-AI-Diffusion offers an optimized text-to-image generation model based on Stable Diffusion, capable of producing high-quality images in modern art styles from both Chinese and English text prompts. It is targeted at users seeking advanced AI art generation with bilingual input capabilities.

How It Works

The project comprises two main components: an optimized text encoder and a diffusion model. The text encoder, SkyCLIP, is a distilled, bilingual (Chinese/English) CLIP model trained efficiently using text data. This approach significantly reduces data and computational requirements for reproduction and fine-tuning. The diffusion model is fine-tuned from stable-diffusion-v1.5, with prompts augmented by the tag 'sai-v1 art' to guide the learning of specific styles and quality.

Quick Start & Requirements

• Install via diffusers library.
• Requires a CUDA-enabled GPU (training utilized 16x A100s).
• Example usage provided in the README.

Highlighted Details

• Supports Chinese, English, and mixed-language prompts.
• Generates images in modern art styles.
• Compatible with stable_diffusion_1.x official models and fine-tuned variants.
• SkyCLIP model offers efficient bilingual CLIP training and evaluation on Flickr30K-CN.

Maintenance & Community

• Project is under continuous development and optimization.
• WeChat QR code provided for joining a developer group.

Licensing & Compatibility

• License: CreativeML Open RAIL-M.
• This license permits commercial use but may have specific use-case restrictions.

Limitations & Caveats

The model is still under continuous optimization, with the expectation of more stable updates in the future.