Lag-Llama introduces the first open-source foundation model for probabilistic time series forecasting, enabling zero-shot forecasting and fine-tuning for various frequencies and prediction lengths. It targets researchers and practitioners seeking robust time series prediction capabilities without extensive dataset-specific training.

How It Works

Lag-Llama is a probabilistic forecasting model that outputs a probability distribution for each predicted timestep. Its architecture is based on a transformer, leveraging causal attention and rotary positional embeddings (RoPE). The model's strength lies in its foundation model approach, allowing it to generalize across different time series frequencies and prediction horizons with minimal or no task-specific fine-tuning.

Quick Start & Requirements

• Install: Primarily through Python packages. The README recommends updating requirements.
• Prerequisites: Python environment. Specific versions are not explicitly stated but recent package updates suggest Python 3.8+ is likely. GPU is recommended for performance.
• Resources: Colab demos are provided for zero-shot forecasting and preliminary fine-tuning.
• Links: Colab Demo 1, Colab Demo 2, Paper, Model Weights

Highlighted Details

• Zero-shot forecasting on datasets of any frequency and prediction length.
• Fine-tuning scripts are available to replicate paper experiments.
• Recommendations for tuning context length and learning rate for optimal performance.
• Critical fixes to kv_cache implementation for improved accuracy.

Maintenance & Community

• Active development with recent updates addressing critical issues and providing reproduction scripts.
• Contact provided via email for specific questions; GitHub issues recommended for code/usage problems.
• GitHub Repository

Licensing & Compatibility

• The README does not explicitly state a license. Model weights are available on Hugging Face, typically under a permissive license unless otherwise specified.

Limitations & Caveats

The project is still evolving, with one Colab demo noted as "preliminary." Best practices for zero-shot prediction and fine-tuning are emphasized, suggesting that default configurations may not yield optimal results without tuning context length and learning rate.