This repository provides a collection of LogitsProcessor implementations designed to fine-tune the behavior of Large Language Models (LLMs) for specific tasks beyond basic text generation. It targets developers and researchers working with LLMs who need to enforce constraints, guide output format, or influence response content. The primary benefit is enabling more controlled and predictable LLM outputs for applications like Q&A, code generation, and structured data extraction.

How It Works

The library offers specialized LogitsProcessor classes that intercept and modify the probability distribution of the next token during LLM generation. This is achieved by integrating with popular LLM inference frameworks like transformers and vLLM. For instance, GenLengthLogitsProcessor adjusts EOS token probabilities to control output length, while ForceLastPhraseLogitsProcessor ensures specific phrases precede the end of generation, useful for structured outputs.

Quick Start & Requirements

• Install via pip: pip install logits-processor-zoo
• Supported frameworks: transformers, vLLM, TensorRT-LLM.
• Example usage with vLLM is provided in the README.

Highlighted Details

• GenLengthLogitsProcessor: Encourages or discourages shorter answers by adjusting EOS token likelihood.
• CiteFromPromptLogitsProcessor: Modifies token likelihood based on prompt content to encourage or discourage similarity.
• ForceLastPhraseLogitsProcessor: Guarantees specific phrases (e.g., "References:\n") appear before generation completion.
• MultipleChoiceLogitsProcessor: Guides LLMs to select specific options for multiple-choice questions.
• TriggerPhraseLogitsProcessor: Forces the LLM to generate a specific phrase after encountering a trigger token.

Maintenance & Community

The project is maintained by NVIDIA. Further community engagement details are not explicitly provided in the README.

Licensing & Compatibility

The repository is released under an unspecified license. Compatibility for commercial use or closed-source linking is not detailed.

Limitations & Caveats

The README does not specify the exact license, which may impact commercial adoption. Detailed compatibility notes for all supported frameworks and versions are not provided.