NBCE (Naive Bayes-based Context Extension) is a plug-and-play library that enables any Large Language Model (LLM) to process arbitrarily long contexts without fine-tuning. It is designed for researchers and developers working with LLMs who need to overcome context length limitations for tasks involving extensive documents or conversations.

How It Works

NBCE leverages a formula inspired by Naive Bayes to extend context handling. This approach allows LLMs to effectively process and reason over much longer inputs than their native training limits, achieving linear efficiency with respect to context length. The method is model-agnostic, meaning it can be applied to various LLMs without requiring architectural changes or retraining.

Quick Start & Requirements

• Installation: Typically via pip.
• Prerequisites: Requires a compatible LLM (e.g., OpenBuddy 7B) and sufficient compute resources (e.g., 160GB VRAM for 50k context with a 7B model on 8 A800 GPUs).
• Demo: Available for testing with provided context and questions.

Highlighted Details

• Enables LLMs to handle contexts exceeding 50k tokens (e.g., 7B model with 160GB VRAM).
• "Plug-and-play" functionality requires no model fine-tuning.
• Achieves linear efficiency with respect to context length.
• Demonstrated success in tasks like resume matching and essay scoring.

Maintenance & Community

• The project is associated with authors from NAACL 2024.
• Community support is available via QQ group: 808623966.

Licensing & Compatibility

• The README does not explicitly state a license. Further clarification is needed for commercial use or integration into closed-source projects.

Limitations & Caveats

The project is described as having "decent results" and is suitable for experimentation, suggesting it may not yet offer state-of-the-art performance across all tasks. The exact computational requirements can be substantial, especially for very long contexts.