metaflow  by Netflix

Framework for building and managing AI/ML systems

Created 6 years ago
9,469 stars

Top 5.4% on SourcePulse

GitHubView on GitHub
Project Summary

Metaflow is a Python framework for building, managing, and deploying AI/ML systems, designed for scientists and engineers. It streamlines the entire ML lifecycle from prototyping to production, enabling rapid iteration and robust system delivery.

How It Works

Metaflow unifies code, data, and compute, providing a Pythonic API for rapid local prototyping, experiment tracking, versioning, and visualization. It supports horizontal and vertical scaling in the cloud, utilizing CPUs and GPUs with fast data access for parallel and gang-scheduled workloads. It also offers dependency management and one-click deployment to production orchestrators with reactive orchestration support.

Quick Start & Requirements

  • Install via pip: pip install metaflow or conda: conda install -c conda-forge metaflow.
  • Recommended: Follow the tutorial for a step-by-step guide.
  • For scaling and production features, configure cloud infrastructure.

Highlighted Details

  • Supports rapid local prototyping and notebook integration.
  • Built-in experiment tracking, versioning, and visualization.
  • Scales horizontally and vertically in the cloud, supporting CPUs and GPUs.
  • Offers one-click deployment to production orchestrators with reactive orchestration.

Maintenance & Community

  • Originally developed at Netflix, now supported by Outerbounds.
  • Active community on Slack.
  • Contributions are welcome via the contribution guide.

Licensing & Compatibility

  • Apache License 2.0.
  • Permissive license suitable for commercial use and integration with closed-source projects.

Limitations & Caveats

The README does not detail specific limitations or known issues. Advanced features require cloud infrastructure configuration.

Health Check
Last Commit

21 hours ago

Responsiveness

1 day

Pull Requests (30d)
53
Issues (30d)
19
Star History
78 stars in the last 30 days

Explore Similar Projects

Starred by Luis Capelo Luis Capelo(Cofounder of Lightning AI), Addy Osmani Addy Osmani(Head of Chrome Developer Experience at Google), and
23 more.

goose by block

0.8%
20k
Open-source AI agent for automating complex engineering tasks
Created 1 year ago
Updated 20 hours ago
Feedback? Help us improve.