manifests  by kubeflow

Kubernetes manifests for deploying end-to-end ML platforms

Created 6 years ago
952 stars

Top 38.6% on SourcePulse

GitHubView on GitHub
Project Summary

This repository provides community-maintained Kubernetes manifests for deploying the Kubeflow platform. It targets users and engineers seeking to explore and utilize Kubeflow's end-to-end machine learning capabilities across various Kubernetes environments, simplifying complex deployments.

How It Works

The project leverages kustomize to manage and deploy a comprehensive set of Kubeflow components and common services. Manifests are organized into applications (official components), common (dependencies like Istio, Cert-Manager), and experimental modules. It synchronizes upstream component versions, allowing for either a full platform installation or granular component selection, facilitating customized Kubeflow distributions.

Quick Start & Requirements

  • Primary Install Command: while ! kustomize build example | kubectl apply --server-side --force-conflicts -f -; do echo "Retrying to apply resources"; sleep 20; done
  • Prerequisites: Kubernetes v1.32 (master branch), kustomize v5.4.3+, kubectl, kind v0.27+, Docker/Podman, and potential Linux kernel tuning for fs.inotify.
  • Resource Estimates: Recommended 16 GB RAM / 8 CPU cores; can be reduced by excluding components.
  • Documentation: Installation guides, component details, and troubleshooting are available within the repository.

Highlighted Details

  • Supports deployment on popular Kubernetes distributions including Kind, Minikube, EKS, AKS, and GKE.
  • Features Istio CNI for enhanced security and Pod Security Standards compatibility.
  • Offers two Kubeflow Pipelines deployment modes: traditional database-based and Kubernetes-native API.
  • Includes Dex for OIDC authentication, with connectors for various identity providers.
  • Utilizes pre-commit hooks for code quality and integrates security scanning via GitHub Actions.

Maintenance & Community

The project is managed by the Platform/Manifests/security Working Group, with active community engagement on the CNCF Slack channel #kubeflow-platform and biweekly meetings. Contribution guidelines are detailed in CONTRIBUTING.md.

Licensing & Compatibility

The repository's license is not explicitly stated in the provided README content. It is designed for Kubernetes environments and specifies compatibility with major cloud provider managed Kubernetes services.

Limitations & Caveats

The master branch is intended for exploration and feedback; stable releases are recommended for production. Installation commands may require retries due to Kubernetes CRD eventual consistency. Default user credentials must be updated for security-sensitive deployments, and some features like Workspaces 2.0 are still under development.

Health Check
Last Commit

1 day ago

Responsiveness

Inactive

Pull Requests (30d)
15
Issues (30d)
5
Star History
11 stars in the last 30 days

Explore Similar Projects

Starred by Shengjia Zhao Shengjia Zhao(Chief Scientist at Meta Superintelligence Lab), Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), and
14 more.

BIG-bench by google

0.1%
3k
Collaborative benchmark for probing and extrapolating LLM capabilities
Created 4 years ago
Updated 1 year ago
Starred by Aravind Srinivas Aravind Srinivas(Cofounder of Perplexity), Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), and
16 more.

text-to-text-transfer-transformer by google-research

0.1%
6k
Unified text-to-text transformer for NLP research
Created 6 years ago
Updated 5 months ago
Feedback? Help us improve.