clearml-server  by clearml

MLOps backend for experiment/data management, pipelines, orchestration, scheduling & serving

created 6 years ago
422 stars

Top 70.8% on sourcepulse

GitHubView on GitHub
1 Expert Loves This Project
Project Summary

ClearML Server provides a self-hosted backend for ClearML, an MLOps platform for streamlining AI workloads. It targets ML engineers and researchers needing to manage experiments, data, pipelines, and serving, offering a centralized solution for collaboration and workflow automation.

How It Works

The ClearML Server comprises a web application for UI-based experiment management, a RESTful API for logging and querying experiment data, and a file server for storing artifacts like images and models. It supports single-domain or sub-domain configurations, allowing flexible deployment via Docker, cloud AMIs, or Kubernetes. The system leverages MongoDB and Elasticsearch for data storage.

Quick Start & Requirements

  • Install/Run: Deploy via Docker, AWS EC2 AMI, GCP Custom Image, or Kubernetes.
  • Prerequisites: Ports 8080, 8081, and 8008 must be available. Python 3.9 is specified for the client.
  • Configuration: Use clearml-init or manually edit ~/clearml.conf to point the client to the server's api_server, web_server, and files_server.
  • Docs: ClearML Client

Highlighted Details

  • Includes ClearML-Agent Services for long-running jobs like auto-scaling, controllers, and hyperparameter optimization.
  • Supports web login authentication and a non-responsive experiments watchdog.
  • Offers a free hosted service maintained by ClearML.
  • Can be deployed with sub-domains for web app, API, and file storage.

Maintenance & Community

  • Community support via Stack Overflow (tag: clearml) and GitHub Issues for bug reports/feature requests.
  • Contact: clearml@allegro.ai

Licensing & Compatibility

  • License: Server Side Public License v1.0 (SSPL).
  • Compatibility: SSPL may have restrictions on offering ClearML Server as a managed service. Commercial use requires careful review of SSPL terms.

Limitations & Caveats

The ClearML Server relies on MongoDB and Elasticsearch, whose own licensing changes (SSPL, Elastic License) may impact downstream compatibility and commercial use. ClearML-Agent Services configuration is not supported on Windows installations.

Health Check
Last commit

1 week ago

Responsiveness

1 day

Pull Requests (30d)
3
Issues (30d)
5
Star History
13 stars in the last 90 days

Explore Similar Projects

Feedback? Help us improve.