This project provides a novel sampler for Large Language Models (LLMs) designed to prevent repetitive or undesirable phrases ("slop") in generated text. It targets LLM developers and researchers seeking to improve output quality by enforcing custom content constraints, offering a more robust alternative to per-token logit biasing.

How It Works

The AntiSlop sampler employs a backtracking mechanism. When a disallowed phrase is detected in the generated output, the sampler reverts to the point where the phrase began and adjusts token probabilities to avoid that sequence. This approach is advantageous because it operates on full phrases rather than individual tokens, preventing the unintended down-regulation of common word beginnings. It supports string and regex matching for greater flexibility in constraint definition.

Quick Start & Requirements

• API Server:

  git clone https://github.com/sam-paech/antislop-sampler.git && cd antislop-sampler
  pip install fastapi uvicorn ipywidgets IPython transformers bitsandbytes accelerate
  python3 run_api.py --model unsloth/Llama-3.2-3B-Instruct --slop_adjustments_file slop_phrase_prob_adjustments.json
  

• Prerequisites: Python 3.11+ (for Open-WebUI integration), transformers, fastapi, uvicorn, bitsandbytes, accelerate. GPU recommended for model inference.
• Integration: Can be used with Open-WebUI (requires Python 3.11) or directly via the OpenAI-compatible API server.
• Docs: Backtracking visualisation notebook, Generate example notebook

Highlighted Details

• Supports string and regex matching for phrase banning.
• Backtracks and down-regulates probabilities of tokens leading to disallowed phrases.
• Offers an OpenAI-compatible API server for easy integration.
• Can enforce long-range constraints, demonstrated with a JSON validator.
• Default slop list derived from over-represented LLM-generated words.

Maintenance & Community

• Actively developed with recent updates in October 2024.
• Research-grade code, indicating potential for bugs and ongoing development.
• No explicit community links (Discord/Slack) mentioned.

Licensing & Compatibility

• No license is explicitly stated in the README.
• Compatibility with commercial or closed-source projects is uncertain without a license.

Limitations & Caveats

The API is minimalist, lacks concurrency support, and is geared towards local use and testing. Regex bans currently do not support streaming. The default slop list is auto-generated and not well-curated. The implementation is noted as research-grade and may contain bugs.