caliban  by google

CLI tool for reproducible research workflows, locally or in the cloud

Created 5 years ago
500 stars

Top 62.0% on SourcePulse

GitHubView on GitHub
Project Summary

Caliban simplifies research workflows by enabling reproducible, isolated execution of numerical experiments, from local workstations to cloud platforms. It targets machine learning researchers and engineers, facilitating seamless transition from prototype to large-scale cloud deployments with robust job tracking and management.

How It Works

Caliban leverages Docker to create consistent, isolated execution environments for experiments. It builds Docker images based on project dependencies (specified in requirements.txt or setup.py) and executes code within these containers. This approach ensures that experiments run identically across local machines and cloud environments, eliminating "it works on my machine" issues and simplifying GPU/dependency management.

Quick Start & Requirements

  • Install Caliban: pip install caliban
  • Prerequisites: Docker (running), Python >= 3.6.
  • Local MNIST training: git clone https://github.com/google/caliban.git && cd caliban/tutorials/basic && caliban run --nogpu mnist.py
  • Documentation: Read The Docs

Highlighted Details

  • Supports local execution and submission to Google Cloud AI Platform.
  • Enables parameter sweeping via JSON configuration or stdin.
  • Provides caliban shell and caliban notebook for interactive development within the execution environment.
  • Offers caliban cluster for GKE integration and caliban status for job management.

Maintenance & Community

This is a research project, not an official Google product. Issues and contributions are managed via GitHub Issues.

Licensing & Compatibility

Licensed under the Apache License, Version 2.0. Compatible with commercial use.

Limitations & Caveats

Described as a research project with potential bugs and "sharp edges." Cloud submission currently focuses on Google Cloud AI Platform.

Health Check
Last Commit

1 year ago

Responsiveness

1 day

Pull Requests (30d)
0
Issues (30d)
0
Star History
0 stars in the last 30 days

Explore Similar Projects

Starred by Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), Vasek Mlejnsky Vasek Mlejnsky(Cofounder of E2B), and
1 more.

pezzo by pezzolabs

0.2%
3k
Open-source LLMOps platform for streamlining AI workflows
Created 2 years ago
Updated 4 months ago
Starred by Chris Lattner Chris Lattner(Author of LLVM, Clang, Swift, Mojo, MLIR; Cofounder of Modular), Tobi Lutke Tobi Lutke(Cofounder of Shopify), and
12 more.

modular by modular

0.1%
25k
AI toolchain unifying fragmented AI deployment workflows
Created 2 years ago
Updated 7 hours ago
Feedback? Help us improve.