Summary

OpenFugu is an open-source reimplementation of Sakana AI's Fugu LLM orchestrator, enabling users to read, run, train, and serve their own sophisticated LLM coordination mechanisms. It targets researchers and engineers seeking to deploy advanced LLM routing and composition, offering a powerful alternative to closed-source solutions.

How It Works

The core mechanism uses a small coordinator model (TRINITY or Fugu-Ultra's Conductor) to route queries to a pool of larger LLMs. TRINITY employs a ~0.6B backbone with a learned linear head for worker selection, optimized via gradient-free methods. Fugu-Ultra's Conductor generates and executes workflow DAGs for macro-level composition, without modifying worker LLM weights. The system is served via an OpenAI-compatible API.

Quick Start & Requirements

Install via pip install -r requirements.txt. Fetch non-redistributed artifacts (models, data) using python scripts/fetch_artifacts.py. Set environment variables (FUGU_MODEL, FUGU_VECTOR, FUGU_FIXTURE). Live demos require worker LLM API keys. Training the Conductor demands significant GPU resources (e.g., 8x A800-class).

Highlighted Details

• Achieves +107% performance over the best single model via query-level orchestration.
• TRINITY coordinator trains from scratch in minutes using gradient-free optimization.
• Fugu-Ultra supports recursive workflows and adaptive k-of-n worker pool selection.
• Offers an OpenAI-compatible API endpoint for serving orchestrated LLMs.

Maintenance & Community

This is an independent reimplementation, not affiliated with Sakana AI. The README provides no specific community links (Discord, Slack) or roadmap details.

Licensing & Compatibility

OpenFugu code is Apache-2.0 licensed. However, trained weights (e.g., openfugu-conductor-3b) are subject to the Llama 3.2 Community License, potentially restricting commercial use. Third-party materials are fetched, not redistributed.

Limitations & Caveats

Users must manually fetch third-party artifacts and models due to licensing. Training the Conductor requires substantial GPU resources. Performance claims have caveats (e.g., query-level vs. per-step routing, potential overfitting). This is a reverse-engineered implementation.