This project provides a self-hostable web interface for the ChatGLM language model, aiming to replicate the user experience of ChatGPT. It's designed for users who want to run a powerful conversational AI locally, offering offline capabilities and the flexibility to use custom-tuned GLM models.

How It Works

The project utilizes FastAPI for the backend API and Vue 3 for the frontend. It supports streaming output from ChatGLM models, allowing users to adjust parameters, manage conversation history, and save outputs as images. The architecture is forked from existing popular ChatGPT web UIs, incorporating features like knowledge base Q&A, though some advanced features from the original repositories are still under development.

Quick Start & Requirements

• Backend: python main.py (with optional arguments for device, quantization, host, port). Requires Python 3.8+.
• Frontend: pnpm bootstrap then pnpm dev. Requires Node.js (v16 or v18 recommended).
• Dependencies: pnpm for frontend, requirements.txt for backend.
• Hardware: GPU with 6GB VRAM (INT4 quantization) to 13GB VRAM (FP16) for inference.
• Knowledge Base: Run python gen_data.py before starting the API.
• Docs: https://github.com/NCZkevin/chatglm-web

Highlighted Details

• Supports ChatGLM-6B and potentially other models.
• Offers offline and self-deployable operation.
• Features include streaming output, parameter tuning, and context selection.
• Includes a knowledge base Q&A functionality.

Maintenance & Community

The project is a fork of Chanzhaoyu/chatgpt-web and WenJing95/chatgpt-web. Contributions are guided by a contribution guide.

Licensing & Compatibility

MIT License. Permissive for commercial use and integration with closed-source applications.

Limitations & Caveats

Some features from parent repositories (e.g., permissions, prompt store, Langchain integration) are marked as "to be implemented." Docker deployment instructions are pending. The project relies on specific Node.js versions and may require manual configuration for certain frontend behaviors like typewriter effects when behind reverse proxies.