This project provides practical examples of deploying machine learning models within Apache Kafka Streams for real-time, scalable production environments. It targets data scientists and engineers seeking to integrate Python, TensorFlow, Keras, and H2O models into robust streaming pipelines.

How It Works

The examples demonstrate deploying pre-trained models (e.g., H2O GBM, TensorFlow CNN, DL4J Iris) as Kafka Streams applications. This approach leverages Kafka's distributed nature for scalability and fault tolerance, enabling real-time inference on streaming data. The project emphasizes practical integration, including unit tests for validation and guidance on running applications with a local Kafka cluster.

Quick Start & Requirements

• Install: Download the project and run mvn clean package.
• Prerequisites: Java 8, Maven 3, Apache Kafka 2.5 (compatible with 1.1 and 2.x).
• Setup: No additional configuration needed for building and running unit tests, as models are included and Kafka helpers are provided. Running main classes requires a local Kafka cluster and topic creation.
• Links: Blog posts and slide decks are available for theoretical background.

Highlighted Details

• Demonstrates deployment of H2O GBM, H2O Deep Learning, TensorFlow CNN, DL4J, and Keras models.
• Includes unit tests for each example.
• Models are included in the repository, increasing download size but simplifying setup.
• Mentions integration with TensorFlow Serving and gRPC for more advanced use cases.

Maintenance & Community

No specific contributors, sponsorships, or community links (Discord/Slack) are mentioned in the README.

Licensing & Compatibility

The README does not explicitly state a license. Compatibility is noted for Kafka/Kafka Streams 1.1 and 2.x.

Limitations & Caveats

The project is not tested on Windows due to Kafka's limitations on the platform. The examples are described as "very simple and lightweight."