77 Open Source Apache Kafka Software Projects
Free and open source apache kafka code projects including engines, APIs, generators, and tools.
Debezium 3765 ⭐
Change data capture for a variety of databases. Please log issues at https://issues.redhat.com/browse/DBZ.
Oryxproject Oryx 1722 ⭐
Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning
Pykafka 1045 ⭐
Apache Kafka client for Python; high-level & low-level consumer/producer, with great performance.
Kowl 889 ⭐
Kafka WebUI for exploring messages, consumers. configurations and more with a focus on a good UI & UX.
Kafka Storm Starter 729 ⭐
Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.
Cp Helm Charts 483 ⭐
The Confluent Platform Helm charts enable you to deploy Confluent Platform services on Kubernetes for development, test, and proof of concept environments.
Wirbelsturm 330 ⭐
Wirbelsturm is a Vagrant and Puppet based tool to perform 1-click local and remote deployments, with a focus on big data tech like Kafka.
Azkarra Streams 119 ⭐
🚀 Azkarra is a lightweight java framework to make it easy to develop, deploy and manage cloud-native streaming microservices based on Apache Kafka Streams.
Tensorflow Serving Java Grpc Kafka Streams 97 ⭐
Kafka Streams + Java + gRPC + TensorFlow Serving => Stream Processing combined with RPC / Request-Response
Kafka Streams Real Time Stream Processing 87 ⭐
This is the central repository for all materials related to Kafka Streams : Real-time Stream Processing! Book by Prashant Pandey.
Slimmessagebus 81 ⭐
Lightweight message bus interface for .NET (pub/sub and request-response) with transport plugins for popular message brokers.
Cp All In One 104 ⭐
docker-compose.yml files for cp-all-in-one , cp-all-in-one-community, cp-all-in-one-cloud
Collabnix Pico 59 ⭐
Object Detection and Analysis Made easy using Raspberry Pi, Apache Kafka, AWS Rekognition & Docker
Spark Twitter Sentiment Analysis 38 ⭐
Sentiment Analysis of a Twitter Topic with Spark Structured Streaming
Storm Dynamic Spout 38 ⭐
A framework for building spouts for Apache Storm and a Kafka based spout for dynamically skipping messages to be processed later.
Opensourcecomputervision 34 ⭐
Open Source Computer Vision with TensorFlow, MiniFi, Apache NiFi, OpenCV, Apache Tika and Python For processing images from IoT devices like Raspberry Pis, NVidia Jetson TX1, NanoPi Duos and more which are equipped with attached cameras or external USB webcams, we use Python to interface via OpenCV and PiCamera. From there we run image processing at the edge on these IoT device using OpenCV and TensorFlow to determine attributes and image analytics. A pache MiniFi coordinates running these Python scripts and decides when and what to send from that analysis and the image to a remote Apache NiFi server for additional processing. At the Apache NiFi cluster in the cluster it routes the images to one processing path and the JSON encoded metadata to another flow. The JSON data (with it's schema referenced from a central Schema Registry) is routed and routed using Record Processing and SQL, this data in enriched and augment before conversion to AVRO to be send via Apache Kafka to SAM. Streaming Analytics Manager then does deeper processing on this stream and others including weather and twitter to determine what should be done on this data. References https://community.hortonworks.com/articles/103863/using-an-asus-tinkerboard-with-tensorflow-and-pyth.html https://community.hortonworks.com/articles/118132/minifi-capturing-converting-tensorflow-inception-t.html https://github.com/tspannhw/rpi-noir-screen https://community.hortonworks.com/articles/77988/ingest-remote-camera-images-from-raspberry-pi-via.html https://community.hortonworks.com/articles/107379/minifi-for-image-capture-and-ingestion-from-raspbe.html https://community.hortonworks.com/articles/58265/analyzing-images-in-hdf-20-using-tensorflow.html
Learn Hadoop And Spark 30 ⭐
This repository focuses on gathering and making a curated list resources to learn Hadoop for FREE.
Docker Kafka Zookeeper 27 ⭐
Docker image for Kafka (0.10.x - 0.11.x - 1.x, 2.x) message broker including Zookeeper
Franzy 21 ⭐
A set of Clojure libraries to work with Apache Kafka (producer, consumer, rebalancing, administration, and validation).
Confluent Avro Py 20 ⭐
An Avro SerDe implementation that integrates with the confluent schema registry and serializes and deserializes data according to the defined confluent wire format
Ravine 18 ⭐
A bridge between synchronous HTTP workload and event-driven landscapes powered by Apache Kafka.
Kafka_producer_consumer_tutorial 12 ⭐
A tutorial on building a real-time data streaming application pipeline with Apache Kafka🔥🔥🔥
Zeelos 10 ⭐
A prototype of utilizing Apache Kafka and Lightweight M2M protocol as the backbone for cloud/edge IoT integration.