Seldon Core is an open-source MLOps framework designed for deploying, managing, and monitoring machine learning models at scale on Kubernetes. It targets ML engineers and data scientists who need to productionize models, offering features like A/B testing, explainability, and advanced metrics, with over 2 million installs across various organizations.

How It Works

Seldon Core transforms ML models (TensorFlow, PyTorch, scikit-learn, etc.) or custom code into production-ready REST/gRPC microservices. It leverages Kubernetes for orchestration and scaling, allowing users to build complex inference graphs composed of predictors, transformers, and routers. This approach provides cloud-agnostic deployment and integrates advanced ML capabilities directly into the serving layer.

Quick Start & Requirements

• Install: helm install seldon-core seldon-core-operator --repo https://storage.googleapis.com/seldon-charts --namespace seldon-system --set usageMetrics.enabled=true --set istio.enabled=true (requires Helm 3 and kubectl).
• Prerequisites: Kubernetes cluster, Helm 3. Istio is recommended for ingress.
• Setup: Deployment involves applying a SeldonDeployment CRD to Kubernetes.
• Docs: https://docs.seldon.io/
• Notebooks: https://github.com/SeldonIO/seldon-core/tree/master/notebooks

Highlighted Details

• Supports pre-packaged inference servers for popular frameworks (TensorFlow Serving, scikit-learn, XGBoost, MLflow) and language wrappers (Python, Java, R, Node.js, Go).
• Enables complex inference graphs with components like routers, transformers, and combiners for advanced orchestration.
• Provides out-of-the-box features for advanced metrics (Prometheus integration), request logging (ELK integration), explainers, and outlier detectors.
• Cloud-agnostic, tested on AWS EKS, Azure AKS, Google GKE, DigitalOcean, and OpenShift.

Maintenance & Community

• Active development with a large user base and community.
• Slack: https://join.slack.com/t/seldoncommunity/shared_invite/zt-172y7911c-7j4Q40X1_17f403477170
• Working Group: Bi-fortnightly calls.

Licensing & Compatibility

• Apache 2.0 License. Permissive, allowing commercial use and integration with closed-source applications.

Limitations & Caveats

• Seldon Core V2 is available, and V1 documentation is also present, which may cause confusion. Some language wrappers are marked as incubating or ALPHA.