dozer  by getdozer

Real-time data movement tool for CDC from sources to sinks

Created 3 years ago
1,559 stars

Top 26.8% on SourcePulse

GitHubView on GitHub
Project Summary

Dozer is a real-time data movement tool designed for efficiently transferring data from various sources to multiple destinations, primarily data warehouses. It targets engineers and data teams seeking a high-performance alternative to traditional CDC pipelines, offering native support for stateless transformations and significant speed advantages over solutions like Debezium and Kafka.

How It Works

Dozer utilizes Change Data Capture (CDC) to extract data changes from supported sources. It then processes these changes, allowing for stateless transformations before writing them to various sinks. This approach aims for high throughput and low latency, enabling real-time data synchronization and integration.

Quick Start & Requirements

  • Dozer is configured via a single YAML file.
  • Requires a running instance of a supported source database (e.g., PostgreSQL, MySQL) and a sink.
  • Full documentation is available at https://getdozer.io/docs.

Highlighted Details

  • Claims to be "magnitudes of times faster" than Debezium+Kafka.
  • Natively supports stateless transformations.
  • Supports a wide range of sources including PostgreSQL, MySQL, Snowflake, Kafka, MongoDB, S3, GCS, Oracle, and Aerospike.
  • Supports sinks such as ClickHouse, PostgreSQL, MySQL, BigQuery, Oracle, and Aerospike.

Maintenance & Community

  • No specific contributor or community information is detailed in the README.

Licensing & Compatibility

  • The README does not specify a license.

Limitations & Caveats

  • Kafka source extraction is marked as "work in progress" (🚧).
  • MongoDB and S3 source resuming capabilities are marked as "target" (🎯).
  • Oracle and Aerospike sources/sinks are marked as "Enterprise Only."
  • The licensing status is unclear, which may impact commercial use.
Health Check
Last Commit

1 year ago

Responsiveness

Inactive

Pull Requests (30d)
0
Issues (30d)
0
Star History
6 stars in the last 30 days

Explore Similar Projects

Feedback? Help us improve.