This repository provides an API for extensions to the SillyTavern AI chat application, offering features like text summarization, image captioning, text-to-speech, and AI-powered character animation. It targets users who want to enhance their AI interactions with advanced multimedia and processing capabilities.

How It Works

The project runs as a separate Python server that SillyTavern connects to via an API URL. It leverages various machine learning models for its functionalities, with options to utilize CPU or GPU (CUDA/MPS) for acceleration. Key modules include text summarization using BART, image captioning with BLIP, and AI animation via the Talkinghead module.

Quick Start & Requirements

• Install: Use Conda (recommended) or vanilla Python.
  • Conda: conda create -n extras python=3.11, conda activate extras, pip install -r requirements.txt, python server.py --enable-modules=caption,summarize,classify
  • Vanilla Python: python -m pip install -r requirements.txt, python server.py --enable-modules=caption,summarize,classify
• Prerequisites: Python 3.11, Git, Conda (recommended). Specific modules may require additional setup (e.g., wxpython for Talkinghead manual poser, CUDA for GPU acceleration).
• Resources: Approximately 6 GB RAM for most modules; Stable Diffusion adds more. GPU acceleration is highly recommended for performance.
• Docs: https://docs.sillytavern.app/
• Colab: https://colab.research.google.com/github/SillyTavern/SillyTavern/blob/release/colab/GPU.ipynb

Highlighted Details

• Project officially discontinued as of April 24, 2024; most modules are now native to SillyTavern.
• Supports CPU and GPU (CUDA, MPS) acceleration for various modules.
• Offers extensive module options including TTS (Coqui, Silero, Edge), image generation (Stable Diffusion), and AI animation (Talkinghead).
• Includes a web search module using Google or DuckDuckGo.

Maintenance & Community

• The project is discontinued and will not receive new updates or modules.
• Community support may be limited for issues.

Licensing & Compatibility

• AGPLv3 license.

Limitations & Caveats

• The project is discontinued, meaning no new features or bug fixes are expected.
• Coqui TTS is noted as unstable and may cause compatibility issues.
• The chromadb module for the deprecated Smart Context extension is likely unnecessary.