This project provides a Python-based framework for deploying and managing AI workloads across distributed infrastructure, including bare-metal servers and cloud GPUs. It targets developers seeking a simplified, serverless experience for AI inference and computation, offering autoscaling, secure communication, and fleet management.

How It Works

Beta9 utilizes Python decorators (@endpoint, @function, @task_queue, @schedule) to define and deploy AI workloads. These workloads are containerized and exposed as load-balanced HTTP endpoints. The system manages distributed clusters, allowing users to connect arbitrary GPU-enabled machines via a simple agent installation script. It leverages a Tailscale-powered service mesh for secure, encrypted communication (WireGuard) and provides a centralized control plane for fleet management.

Quick Start & Requirements

• Local Setup: make setup (requires Docker, k3d).
• SDK: Python 3.8+. make setup-sdk to install.
• Bare-metal: Requires running a provided agent script on the target machine.
• Local DNS: Optional, requires dnsmasq for local domain resolution.
• Docs: SDK Readme

Highlighted Details

• Serverless AI workloads with Python decorators.
• Autoscaling and scale-to-zero capabilities.
• Connect bare-metal GPU nodes with a single command.
• Tailscale-powered service mesh for secure, encrypted communication.

Maintenance & Community

• Community support via Slack and GitHub issues.
• Active development indicated by frequent updates and multiple language READMEs.

Licensing & Compatibility

• The README does not explicitly state a license. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The project is in "beta9," suggesting potential instability or breaking changes. The lack of a specified license poses a significant adoption blocker for commercial use.