This project provides a collection of one-click installers and packaged programs for self-hosting various AI models, including large language models and image generation tools. It targets users who want to easily deploy and experiment with these AI technologies locally, offering OpenAI API compatibility for seamless integration with existing applications.

How It Works

The project bundles pre-configured environments for popular open-source AI projects like llama.cpp, ChatGLM, and stable-diffusion-webui. It simplifies the setup process by providing downloadable packages with integrated models and scripts, abstracting away complex dependency management and configuration. The core benefit is rapid deployment of powerful AI capabilities on local hardware.

Quick Start & Requirements

• Installation: Download packaged files from the "release" section. Use update.bat for updates.
• Prerequisites:
  • ngrok_token environment variable for exposing the API.
  • Models for llama.cpp and stable-diffusion-webui need to be downloaded separately and placed in designated directories.
  • stable-diffusion-webui requires at least one model for startup.
• Resources: Specific hardware requirements depend on the chosen AI model (e.g., GPU for Stable Diffusion, significant RAM for LLMs).
• Links:
  • llama.cpp model download: WSL or Linux environment via get_llama_model.sh.
  • stable-diffusion-webui installer: https://github.com/AUTOMATIC1111/stable-diffusion-webui/releases/download/v1.0.0-pre/sd.webui.zip
  • stable-diffusion-webui installation: https://github.com/AUTOMATIC1111/stable-diffusion-webui#installation-and-running
  • text-generation-webui installers: https://github.com/oobabooga/text-generation-webui#one-click-installers

Highlighted Details

• OpenAI API compatibility for easy integration.
• Includes one-click installers for stable-diffusion-webui and text-generation-webui.
• Bundles llama.cpp with a modified Unicode-supporting Windows command-line interface.
• Offers a packaged program for the Wenda LLM.

Maintenance & Community

• No specific information on contributors, sponsorships, or community channels (Discord/Slack) is provided in the README.

Licensing & Compatibility

• ChatGLM is Apache-2.0 licensed.
• Other project licenses are not explicitly stated but are assumed to align with the respective upstream projects. Compatibility for commercial use depends on the individual components' licenses.

Limitations & Caveats

The README mentions a "trimmed version" of chatglm-6b-int4 and requires users to download models separately for llama.cpp and stable-diffusion-webui, indicating potential setup complexity beyond the "one-click" promise. The Wenda package download link requires a Baidu Netdisk account.