Summary

FLUID (Flexible Unidirectional Inference Diffusion) is a framework for efficiently adapting pre-trained Autoregressive (AR) Large Language Models (LLMs) into parallel diffusion models. It targets researchers and practitioners seeking to leverage LLM priors for diffusion tasks with significantly reduced training data requirements, offering state-of-the-art performance through novel causal and dynamic horizon modeling techniques.

How It Works

FLUID introduces two core innovations: Strictly Causal Alignment employs a lower-triangular attention mask within the Transformer backbone, preserving the inductive biases of AR models and enabling seamless initialization from GPT-style checkpoints, unlike standard bidirectional diffusion. Elastic Horizon Modeling dynamically modulates denoising strides ($K_t$) based on local information density, using an entropy-driven mechanism to "sprint" through predictable text and "downshift" for complex reasoning, addressing the entropy-horizon dilemma. This approach allows for efficient adaptation with orders of magnitude less training data.

Quick Start & Requirements

• Installation: Clone the repository (git clone https://github.com/Oli-lab-nun/FLUID.git), navigate to the LLaMA-Factory directory, and install using pip install -e .[torch,metrics].
• Prerequisites: Requires PyTorch and dependencies managed by LLaMA-Factory. Training utilizes LoRA. The base model is openPangu-Embedded-7B.
• Resources: Official implementation for a research paper submission (ACL 2026).
• Links: Code repository: https://github.com/Oli-lab-nun/FLUID.git. Model weights available on Hugging Face (link not directly provided in README).

Highlighted Details

• Training Efficiency: Achieves state-of-the-art results on GSM8K (91.9) and MATH500 (61.8) using only 2.7B tokens of adaptation data, significantly outperforming models trained on trillions of tokens.
• Performance: FLUID-7B matches or exceeds top-tier AR and Diffusion baselines across benchmarks like MMLU, GSM8K, MATH500, and HumanEval.
• LLaMA-Factory Integration: Fully compatible with the LLaMA-Factory ecosystem for efficient LoRA fine-tuning and scaling.

Maintenance & Community

The README does not provide specific details regarding maintainers, community channels (e.g., Discord/Slack), roadmaps, or sponsorship information.

Licensing & Compatibility

FLUID is adapted from the openPangu-Embedded-7B model. The README directs users to consult the original openPangu repository and its license files for detailed terms, implying FLUID's licensing is contingent upon openPangu's. Specific licensing for FLUID itself is not explicitly stated.

Limitations & Caveats

As an official implementation for a research paper submission (ACL 2026), FLUID may be primarily oriented towards research use cases. No specific technical limitations, unsupported platforms, or known bugs are detailed in the provided README.