Analyzing Real-time Order Deliveries using CDC with Debezium and Pinot

00:00:00 Intro 00:02:00 Kenny Bastani Intro 00:04:02 Gunnar Morling Intro 00:05:00 Database-related Humor 00:06:20 Open Source Database Adoption 00:09:30 Real-time Delivery Example & Demo 00:31:50 OLAP + OLTP for Microservices Lessons Learned 00:34:31 Debezium for change data capture and outbox pattern 00:40:48 Outbox Pattern 00:45:40 Demo 00:60:00 Q&A Wouldn’t it be great if your analytics system was updated within seconds after data changes in your production database? Applying data transformations and yielding aggregated actionable insights in near real-time? Turns out, that is all doable with open-source software. In this fast-paced session, we’ll cover the benefits of combining Apache Pinot and Debezium to power (near) real-time OLAP use cases, and marvel at how these two technologies come together in a real-world example simulating an order delivery service — from microservice orchestration to data visualization! We’ll walk you through solutions to common challenges like capturing domain events from production databases, applying non-trivial processing on streaming data, and serving queries at a large scale with low latency. Come and join us for a demo-centric presentation about two open-source projects, which used together can bring the value of your data to the next level: - Apache Pinot for scalable real-time analytics - Debezium for change data capture and outbox pattern -------------------------------------- Speakers: Kenny Bastani As an open source contributor and blogger, Kenny enjoys engaging a community of passionate software developers on topics ranging from graph databases, microservices, and cloud-native applications. Kenny is also a regular speaker at industry conferences such as OSCON, SpringOne Platform, and GOTO. He maintains a personal blog about software architecture at kennybastani.com with tutorials and open source reference examples for building event-driven microservices and serverless architectures. Gunnar Morling Gunnar is a software engineer and open-source enthusiast by heart. He is leading the Debezium project, a platform for change data capture (CDC). He is a Java Champion, the spec lead for Bean Validation 2.0 (JSR 380) and has founded multiple open source projects such as Deptective and MapStruct. Prior to joining Red Hat, Gunnar worked on a wide range of Java EE projects in the logistics and retail industries. He's based in Hamburg, Germany. ----------------------------------------------------- Resources ----------------------------------------------------- Pinot Meetup https://www.meetup.com/apache-pinot/ Pinot website http://pinot.apache.org/ Pinot docs https://docs.pinot.apache.org/ Pinot Slack https://communityinviter.com/apps/apache-pinot/apache-pinot Pinot Github https://github.com/apache/incubator-pinot Pinot twitter: https://twitter.com/ApachePinot Debezium website https://debezium.io/