Diego Pacheco Tech blog

ksqlDB is an Event Streaming Database. Basically, you turn Kafka Streams(topics) into Databases. Allowing you to have some level of consistency and fast lookups on real-time data. ksqlDB allows you to perform SQL queries on your Kafka topics. It's possible todo pull-based or even push-based queries. Kafka is getting closer and closer to be a Database and that's exciting. To be 100% clear Kafka is not a full-fledge database and I would cherry-pick use cases that fit into the model. There is no one-size-fits-all. So today I want to share a simple POC in a video I made for you guys where we will see push queries in action. So Let's get started!

Kafka Streams is a Streaming library similar to Spark and Flink which works with Apache Kafka. Kafka stream can be useful to process historical and near-real-time big data workloads but also for non-realtime analytical computations at scale for the online world as well. Kafka-Streams is elastic, highly scalable, fault-tolerant, and fully integrated with Kafka. You can use Kafka Stream with Java, Scala, Kotlin JVM applications and also have exactly-once processing semantics. Kafka-Streams is being used by the New York Times, Pinterest, Line, Trivago, Zalando, and many other companies. Today I want to share a video of Kafka-Streams running with Kafka 2.6 and Java JDK 15. So Let's get started. 

There are lots of os problems or use cases that can only address properly addressed today by using streaming techniques. Streaming is needed due to the high volume of data and is almost 100% sure you will need it as your business grow up. Today I want a lecture I recorded last week about the subject. You will see a brief evolution of big data solutions from Hadoop to Modern Streaming with Spark and Flink. You also will see why streaming is needed and why we cannot use traditional solutions for it. I hope you guys like it. Let's go!