RegressLM is a Python library designed for sequence-to-sequence numeric prediction tasks, accommodating any tokenizable input. It empowers machine learning engineers and researchers to pretrain and fine-tune models across diverse applications, such as predicting system performance metrics from unstructured text. The library offers a flexible framework for developing custom numeric prediction solutions.

How It Works

The core of RegressLM is a flexible encoder-decoder architecture adaptable for numeric output. It supports various tokenization strategies and allows for custom vocabulary training. The library integrates with established models like T5Gemma and enables advanced fine-tuning techniques such as LoRA. For long-context scenarios, it incorporates alternative encoders like Mamba-SSM and Performer, offering a novel approach to handling extensive input sequences for prediction.

Quick Start & Requirements

• Installation: Core libraries: pip install -e .. For T5Gemma/LoRA extras: pip install ".[extras]". Installation is estimated to take under a minute.
• Prerequisites: Python 3.10+ is required. Linux is strongly preferred for deep learning workloads.
• Resources: Example Colabs are available for synthetic density training, Triton GPU kernel latency prediction, and Kaggle experiment outcome prediction.

Highlighted Details

• Supports T5Gemma (V1+V2) encoder-decoder architectures and end-to-end T5Gemma baselines.
• Enables long-context processing (100K+ tokens) via Mamba-SSM and Performer encoder integrations.
• Facilitates multi-objective prediction by decoding concatenated token sequences.
• Allows training custom vocabularies from offline corpora for specialized tokenization.

Maintenance & Community

Core contributors include Xingyou Song, Yash Akhauri, Jiyoun Ha, and Bryan Lewandowski. No other community channels or maintenance indicators are detailed in the README.

Licensing & Compatibility

The README does not specify a license type. This omission requires clarification regarding its usability for commercial or closed-source projects.

Limitations & Caveats

The project is explicitly stated as "not an officially supported Google product." Specific limitations regarding supported model architectures beyond those mentioned (T5Gemma, Mamba, Performer) or potential compatibility issues are not detailed.