caliban  by google

CLI tool for reproducible research workflows, locally or in the cloud

created 5 years ago
501 stars

Top 62.9% on sourcepulse

GitHubView on GitHub
Project Summary

Caliban simplifies research workflows by enabling reproducible, isolated execution of numerical experiments, from local workstations to cloud platforms. It targets machine learning researchers and engineers, facilitating seamless transition from prototype to large-scale cloud deployments with robust job tracking and management.

How It Works

Caliban leverages Docker to create consistent, isolated execution environments for experiments. It builds Docker images based on project dependencies (specified in requirements.txt or setup.py) and executes code within these containers. This approach ensures that experiments run identically across local machines and cloud environments, eliminating "it works on my machine" issues and simplifying GPU/dependency management.

Quick Start & Requirements

  • Install Caliban: pip install caliban
  • Prerequisites: Docker (running), Python >= 3.6.
  • Local MNIST training: git clone https://github.com/google/caliban.git && cd caliban/tutorials/basic && caliban run --nogpu mnist.py
  • Documentation: Read The Docs

Highlighted Details

  • Supports local execution and submission to Google Cloud AI Platform.
  • Enables parameter sweeping via JSON configuration or stdin.
  • Provides caliban shell and caliban notebook for interactive development within the execution environment.
  • Offers caliban cluster for GKE integration and caliban status for job management.

Maintenance & Community

This is a research project, not an official Google product. Issues and contributions are managed via GitHub Issues.

Licensing & Compatibility

Licensed under the Apache License, Version 2.0. Compatible with commercial use.

Limitations & Caveats

Described as a research project with potential bugs and "sharp edges." Cloud submission currently focuses on Google Cloud AI Platform.

Health Check
Last commit

1 year ago

Responsiveness

Inactive

Pull Requests (30d)
0
Issues (30d)
0
Star History
2 stars in the last 90 days

Explore Similar Projects

Starred by Aravind Srinivas Aravind Srinivas(Cofounder of Perplexity), Chip Huyen Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and
8 more.

higgsfield by higgsfield-ai

0.3%
3k
ML framework for large model training and GPU orchestration
created 7 years ago
updated 1 year ago
Starred by Chip Huyen Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), Ben Firshman Ben Firshman(Cofounder of Replicate), and
6 more.

Made-With-ML by GokuMohandas

0.4%
41k
ML course for production-grade applications
created 6 years ago
updated 11 months ago
Feedback? Help us improve.