Robert Metzger - Tiny Flink — Minimizing the memory footprint of Apache Flink

Apache Flink has been designed for, and is mostly used with large-scale real-time data processing use-cases. Companies report about TBs of data being processed per second, or TBs of state in huge clusters. But what if you need to process low-throughput streams? Running a full, distributed Flink cluster might be an overkill, as there’s quite a bit of overhead for distributed coordination. In this talk, we’ll explore options to reduce your resource footprint. We’ll dive deeper into Flink’s MiniCluster, allowing you to run Flink in-JVM for integration tests, as a micro service or just a small processing your data in Kubernetes. We will also discuss lessons learned from running MiniCluster in production for a service offering Flink SQL in the cloud. Attend this talk if you want to learn about Apache Flink and its various options to deploy and configure it. Speaker: Robert Metzger More: https://2023.berlinbuzzwords.de/sessions/?id=BWNJZN Web: https://2023.berlinbuzzwords.de/ Fediverse: https://floss.social/@berlinbuzzwords Linkedin: https://www.linkedin.com/showcase/13978964/ Twitter: https://twitter.com/berlinbuzzwords