metaflow  by Netflix

Framework for building and managing AI/ML systems

Created 6 years ago
9,702 stars

Top 5.2% on SourcePulse

GitHubView on GitHub
Project Summary

Metaflow is a Python framework for building, managing, and deploying AI/ML systems, designed for scientists and engineers. It streamlines the entire ML lifecycle from prototyping to production, enabling rapid iteration and robust system delivery.

How It Works

Metaflow unifies code, data, and compute, providing a Pythonic API for rapid local prototyping, experiment tracking, versioning, and visualization. It supports horizontal and vertical scaling in the cloud, utilizing CPUs and GPUs with fast data access for parallel and gang-scheduled workloads. It also offers dependency management and one-click deployment to production orchestrators with reactive orchestration support.

Quick Start & Requirements

  • Install via pip: pip install metaflow or conda: conda install -c conda-forge metaflow.
  • Recommended: Follow the tutorial for a step-by-step guide.
  • For scaling and production features, configure cloud infrastructure.

Highlighted Details

  • Supports rapid local prototyping and notebook integration.
  • Built-in experiment tracking, versioning, and visualization.
  • Scales horizontally and vertically in the cloud, supporting CPUs and GPUs.
  • Offers one-click deployment to production orchestrators with reactive orchestration.

Maintenance & Community

  • Originally developed at Netflix, now supported by Outerbounds.
  • Active community on Slack.
  • Contributions are welcome via the contribution guide.

Licensing & Compatibility

  • Apache License 2.0.
  • Permissive license suitable for commercial use and integration with closed-source projects.

Limitations & Caveats

The README does not detail specific limitations or known issues. Advanced features require cloud infrastructure configuration.

Health Check
Last Commit

5 days ago

Responsiveness

1 day

Pull Requests (30d)
16
Issues (30d)
8
Star History
45 stars in the last 30 days

Explore Similar Projects

Feedback? Help us improve.