debezium-examples  by debezium

Change Data Capture examples for diverse data streaming and integration

Created 9 years ago
1,672 stars

Top 24.5% on SourcePulse

GitHubView on GitHub
1 Expert Loves This Project
Project Summary

This repository provides a comprehensive collection of examples for Debezium, a leading Change Data Capture (CDC) platform. It targets developers and architects seeking practical, ready-to-deploy configurations and code snippets to implement real-time data streaming and integration solutions. The benefit lies in accelerating Debezium adoption by offering diverse, use-case-driven demonstrations.

How It Works

The examples showcase Debezium's core capability: capturing row-level changes from databases (including PostgreSQL, MySQL, MongoDB, Oracle, and SQL Server) and streaming them as events. The repository demonstrates various architectural patterns and integration scenarios, leveraging Debezium Server and Kafka Connect. Approaches include streaming changes to Kafka, Apache Pulsar, Google Cloud Pub/Sub, and Amazon Kinesis, enabling downstream processing for analytics, microservices, caching, and more. Key examples highlight patterns like Retrieval-Augmented Generation (RAG), Audit Logs, Cache Invalidation, the Outbox Pattern, and distributed Saga transactions.

Quick Start & Requirements

Getting started involves exploring the provided tutorial example. The repository includes Docker Compose files and OpenShift templates, suggesting containerized deployments. Specific prerequisites vary by example but generally include Debezium components, target databases, and messaging systems (e.g., Kafka, Pulsar). Links to official quick-start guides and documentation are recommended for detailed setup.

Highlighted Details

  • Demonstrates AI RAG integration using Debezium Server, Milvus, and Ollama for real-time embedding generation.
  • Provides implementations for critical microservice patterns like the Outbox and Saga patterns for reliable asynchronous communication and distributed transactions.
  • Features examples for high-performance CDC with Quarkus Native Image and integration with graph databases via Infinispan.
  • Covers advanced use cases including GraphQL subscriptions, machine learning pipelines (TensorFlow, Flink), and database-specific features like PostgreSQL TOAST columns and replication slots.

Maintenance & Community

Issues related to Debezium examples should be logged at the official GitHub issue tracker: https://github.com/debezium/dbz/issues. No other community channels or specific contributor details are highlighted in the provided README snippet.

Licensing & Compatibility

The project is licensed under the Apache License, Version 2.0. This license is permissive and generally compatible with commercial use and closed-source applications.

Limitations & Caveats

The repository focuses on demonstrating Debezium's capabilities through examples; individual examples may represent Proof-of-Concept (PoC) level implementations (e.g., JPA Aggregations). The README does not specify overarching limitations for the examples collection itself, but users should evaluate specific examples for production readiness based on their requirements.

Health Check
Last Commit

1 week ago

Responsiveness

Inactive

Pull Requests (30d)
9
Issues (30d)
0
Star History
2 stars in the last 30 days

Explore Similar Projects

Starred by Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), Georgios Konstantopoulos Georgios Konstantopoulos(CTO, General Partner at Paradigm), and
3 more.

risingwave by risingwavelabs

0.2%
9k
Stream processing and serving for AI agents and real-time data applications
Created 4 years ago
Updated 23 hours ago
Starred by Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems") and Chaoyu Yang Chaoyu Yang(Founder of Bento).

seatunnel by apache

0.2%
9k
High-performance multimodal data integration
Created 9 years ago
Updated 1 day ago
Feedback? Help us improve.