This project provides a full-stack ChatGPT clone built entirely in Rust, targeting developers interested in exploring Rust for web applications and AI integration. It simplifies the creation of interactive chatbots using open-source language models and a modern frontend stack.

How It Works

The application leverages Rust for both the backend API serving the language model and the frontend web interface. It utilizes the Rustformers/llm crate for interacting with GGML-formatted language models, enabling local inference. The frontend is built with Leptos and styled using TailwindCSS, offering a reactive and efficient user experience.

Quick Start & Requirements

• Install:
  • rustup toolchain install nightly
  • rustup target add wasm32-unknown-unknown
  • cargo install trunk cargo-leptos
  • npm install -g tailwindcss (or npm install -D tailwindcss for local project)
• Prerequisites: Nightly Rust toolchain, Node.js (for TailwindCSS).
• Model: Download a GGML-formatted model and set MODEL_PATH in .env.
• TailwindCSS: Run npx tailwindcss -i ./input.css -o ./style/output.css --watch.
• Run: cargo leptos watch
• Docs: https://github.com/MoonKraken/rusty_llama

Highlighted Details

• Full-stack Rust implementation for both backend and frontend.
• Uses Rustformers/llm crate for local GGML model inference.
• Frontend built with Leptos framework and styled with TailwindCSS.
• Supports Apple Metal acceleration by default; requires configuration for other GPU setups.

Maintenance & Community

No specific community channels or notable contributors are mentioned in the README.

Licensing & Compatibility

The repository does not explicitly state a license.

Limitations & Caveats

Nvidia CUDA support is mentioned as untested, requiring community contributions for integration. The project relies on specific Rust toolchains and build tools, which may require careful setup.