marmot  by maxpert

Distributed SQLite replication with MySQL compatibility

Created 3 years ago
2,783 stars

Top 16.6% on SourcePulse

GitHubView on GitHub
Project Summary

Marmot provides a leaderless, distributed SQLite replication system designed to overcome the complexity of traditional MySQL active-active setups, especially for edge deployments. It targets users needing a highly available, operationally simple database that integrates seamlessly with the MySQL ecosystem, enabling applications like distributed WordPress with minimal overhead.

How It Works

Marmot utilizes a leaderless architecture with a gossip protocol for cluster membership and anti-entropy. It supports distributed transactions via Percolator-style 2PC, resolving conflicts with Last-Write-Wins (LWW) using HLC timestamps. Replication is handled through row-level Change Data Capture (CDC). Its key differentiator is a MySQL wire-compatible interface, enabling integration with existing MySQL clients, ORMs, and applications without modification.

Quick Start & Requirements

  • Primary Install/Run: Execute the marmot-v2 binary. Start a single-node cluster with ./marmot-v2 or run as a daemon with ./marmot-v2 -daemon.
  • Prerequisites: SQLite must be configured in WAL mode.
  • Links: No direct documentation or quick-start links are provided, but example scripts and a config.toml reference are available.

Highlighted Details

  • MySQL Protocol Compatibility: Connect using standard MySQL clients (mysql CLI, DBeaver, MySQL Workbench) and ORMs.
  • WordPress Support: Fully compatible, enabling distributed multi-region deployments.
  • Leaderless & Gossip-Based: Eliminates single points of failure and complex leader election.
  • Distributed Transactions: Implements 2PC with conflict detection for ACID properties.
  • DDL Replication: Supports distributed schema changes with automatic idempotency and cluster-wide locking.
  • Built-in Vector Search: Integrates local ANN indexes for RAG workloads directly within SQLite.
  • Compact Auto-Increment IDs: Offers 53-bit IDs safe for JavaScript's Number.MAX_SAFE_INTEGER.

Maintenance & Community

Community support is available via Discord and GitHub Discussions. Details on core contributors or sponsorships are not explicitly provided in the README.

Licensing & Compatibility

The project's license is not specified in the README, a critical omission for adoption. Marmot requires SQLite to be in WAL mode for reliable multi-process changes.

Limitations & Caveats

Marmot replicates all tables within a database; selective table replication is not supported. It guarantees eventual consistency, meaning strict serializability across nodes is not provided. Concurrent DDL operations on the same database from multiple nodes are protected by a 30-second lease lock but should generally be avoided. XA transactions are incompatible, and user management (DCL) is local to each node.

Health Check
Last Commit

2 weeks ago

Responsiveness

Inactive

Pull Requests (30d)
0
Issues (30d)
0
Star History
14 stars in the last 30 days

Explore Similar Projects

Starred by Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), Joe Walnes Joe Walnes(Head of Experimental Projects at Stripe), and
9 more.

3FS by deepseek-ai

0.3%
10k
Distributed file system for AI training/inference workloads
Created 1 year ago
Updated 2 weeks ago
Starred by Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems") and Chaoyu Yang Chaoyu Yang(Founder of Bento).

seatunnel by apache

0.2%
9k
High-performance multimodal data integration
Created 8 years ago
Updated 1 day ago
Feedback? Help us improve.