SallyBot is a Discord chatbot designed for users who want to run AI models locally, offering text generation and image creation capabilities without relying on cloud services like ChatGPT. It's suitable for hobbyists and developers interested in integrating LLMs and Stable Diffusion into a Discord bot environment, with a particular emphasis on low-resource hardware like Raspberry Pi.

How It Works

The bot leverages C# and Discord.net to interact with users on Discord. It supports multiple backend AI interfaces, including Oobabooga's Text Generation Web UI (via OpenAI API format), Google's Gemini-Pro API, and the older Dalai project. For image generation, it integrates with Stable Diffusion Web UI. This modular approach allows users to choose their preferred LLM and image generation models, offering flexibility in performance and hardware requirements.

Quick Start & Requirements

• Installation: Clone the repository or download as a ZIP. Requires Visual Studio Community with C#/.NET development modules.
• Discord Bot: Create a Discord bot application, obtain an API key, and enable privileged message intents.
• LLM Integration:
  • Gemini-Pro: Obtain an API key from Google AI Studio.
  • Oobabooga: Install Text Generation Web UI, download models (e.g., 7B GGUF for lower VRAM), and configure one_click.py arguments (--api, --listen-port 7862). Requires 6GB+ VRAM for 7B models.
  • Dalai: Install Dalai, run npx dalai serve (requires specific setup on C: drive with a custom index.js for proper stop command functionality).
• Image Generation: Install Stable Diffusion Web UI, modify webui-user.bat/.sh with --api --xformers, and download models to the appropriate directory. Requires Python 3.10.7 for Stable Diffusion.

Highlighted Details

• Runs LLMs and Stable Diffusion locally, including on Raspberry Pi 4B.
• Supports multiple LLM backends: Oobabooga, Gemini-Pro, Dalai.
• Integrates with Stable Diffusion for local image generation.
• Offers a free tier via Google Gemini-Pro API.

Maintenance & Community

The project appears to be maintained by a single developer (DeSinc). Links to community resources like Discord or Slack are not explicitly provided in the README.

Licensing & Compatibility

The README does not specify a license. Compatibility for commercial use or closed-source linking is not detailed.

Limitations & Caveats

• Dalai integration has known issues with its stop command, requiring a custom workaround.
• Oobabooga integration has specific VRAM requirements (6GB+ for 7B models).
• Stable Diffusion requires a specific Python version (3.10.7) and potential command-line argument adjustments.
• The Dalai project is noted as potentially abandoned.