metaflow  by Netflix

Framework for building and managing AI/ML systems

created 5 years ago
9,325 stars

Top 5.5% on sourcepulse

GitHubView on GitHub
Project Summary

Metaflow is a Python framework for building, managing, and deploying AI/ML systems, designed for scientists and engineers. It streamlines the entire ML lifecycle from prototyping to production, enabling rapid iteration and robust system delivery.

How It Works

Metaflow unifies code, data, and compute, providing a Pythonic API for rapid local prototyping, experiment tracking, versioning, and visualization. It supports horizontal and vertical scaling in the cloud, utilizing CPUs and GPUs with fast data access for parallel and gang-scheduled workloads. It also offers dependency management and one-click deployment to production orchestrators with reactive orchestration support.

Quick Start & Requirements

  • Install via pip: pip install metaflow or conda: conda install -c conda-forge metaflow.
  • Recommended: Follow the tutorial for a step-by-step guide.
  • For scaling and production features, configure cloud infrastructure.

Highlighted Details

  • Supports rapid local prototyping and notebook integration.
  • Built-in experiment tracking, versioning, and visualization.
  • Scales horizontally and vertically in the cloud, supporting CPUs and GPUs.
  • Offers one-click deployment to production orchestrators with reactive orchestration.

Maintenance & Community

  • Originally developed at Netflix, now supported by Outerbounds.
  • Active community on Slack.
  • Contributions are welcome via the contribution guide.

Licensing & Compatibility

  • Apache License 2.0.
  • Permissive license suitable for commercial use and integration with closed-source projects.

Limitations & Caveats

The README does not detail specific limitations or known issues. Advanced features require cloud infrastructure configuration.

Health Check
Last commit

4 days ago

Responsiveness

1 day

Pull Requests (30d)
58
Issues (30d)
9
Star History
612 stars in the last 90 days

Explore Similar Projects

Starred by Aravind Srinivas Aravind Srinivas(Cofounder of Perplexity), Chip Huyen Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and
8 more.

higgsfield by higgsfield-ai

0.3%
3k
ML framework for large model training and GPU orchestration
created 7 years ago
updated 1 year ago
Starred by Chip Huyen Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), Ben Firshman Ben Firshman(Cofounder of Replicate), and
6 more.

Made-With-ML by GokuMohandas

0.4%
41k
ML course for production-grade applications
created 6 years ago
updated 11 months ago
Starred by Nat Friedman Nat Friedman(Former CEO of GitHub), Chip Huyen Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and
7 more.

langflow by langflow-ai

2.9%
93k
Visual tool for AI agent and workflow creation/deployment
created 2 years ago
updated 18 hours ago
Feedback? Help us improve.