PostgreSQL-as-a-Service on K8s at Zalando - #12 DoK Community

// Abstract: PostgreSQL is a powerful, open-source object-relational database system with over 30 years of active development that has earned it a strong reputation for reliability, feature robustness, and performance, but a production-grade deployment requires many complementary technologies to the database core: high availability and automated failover, backup and recovery, monitoring and alerting, centralized access control and logging, connection pooling, and so on. Being not initially accustomed for running stateful workloads, Kubernetes with its infrastructure as a code paradigm, CustomResourceDefinition, and Operator pattern turned out to be extremely convenient for deploying and running PostgreSQL at scale. I will talk about a few open-source project developed and maintained by database team at Zalando which anybody could use to build own PgaaS: 1. https://github.com/zalando/patroni - Tool for PostgreSQL high availability and cluster management. Integrates with K8s API and makes PostgreSQL cloud-native. 2. https://github.com/zalando/spilo - The Docker image that packages Patroni, multiple versions of PostgreSQL, and tools for backup and recovery. 3. https://github.com/zalando/postgres-operator - Implements Kubernetes operator pattern, orchestrates hundreds and thousands of deployments of Patroni/Spilo clusters The aforementioned projects would never get to the current state without an effort of dozens of external contributors. // Bio: During his professional career, Alexander touched PostgreSQL from all possible sides: as a Web Developer, as System Administrator, and as a Database Engineer now. Five years ago he started working on Patroni project and achieved quite a big success with it. Nowadays Patroni is used in the heart of PostgreSQL-as-a-Service on Kubernetes at Zalando to run 1000+ PostgreSQL HA clusters. Besides that, Alexander periodically reports bugs and contributes patches to PostgreSQL and some other open-source projects (usually Postgres related). Connect with Alexander on Twitter: https://twitter.com/cyberdemn Timestamps: 0:00 - Intro to Alexander Kukushkin 0:38 - Patroni/Spilo Background 2:50 - What is it about PostgreSQL that attracts you so much? 5:14 - Advantages of PostgreSQL 8:04 - Zalando's Theory/Ethos 10:13 -Were you contributing to the PostgreSQL community before working with Zalando? 11:48 - Thee Zalando Operators: Patroni, Spilo & PostgreSQL 15:53 - How did you get to the point where you decided to have an operator and why do it as an operator? 18:12 - Where are you at right now? How has that grown? 18:16 - What do you think is gonna happen in the next steps? 21:00 - When do you feel an operator is needed? 24:17 - If at Zalando you have databases that span across clouds and data centres. 24:25 - If you do, What kind of challenges have you had with that? 26:02 - What kind of shortcomings you see in the operators? 26:12 - What enhancement would you like to see as we move forward? 28:39 - Do you feel like these operators are easily customizable? 31:13 - How do you handle rolling cluster updates? 35:26 - Is there a wrong way to approach Replication in order to know what to avoid? 35:29 - Is it just a question of trade-offs? 37:37 - Summary of How Not to Screw Up when Building a High Availability Coaster Talk. 40:35 - Do you have other ideas to look at the cost of HA? 45:11 - Why do people use Oracle? 47:25 - How big does it need for you to get a database as a service? 50:59 - Where does the operator end in the role of a DBA begin? 53:38 - Does the operator handle the failover automatically? - DoK Landscape An interactive landscape of products and services for running data workloads on Kubernetes. (https://dok.community/landscape/) - Videos and Blogs Hundreds of videos and blogs including how tos, tutorials, use cases, case studies, and more. (https://dok.community/landscape/) (https://dok.community/blog/) - DoK Reports DoKC works with an independent research firm to annually produce the industry’s only DoK-focused report. (https://dok.community/dok-reports/) - Livestreams and Local Meetups Weekly livestreams on DoK-related topics including demos, tutorials, visionary talks, panels, and more. (https://www.meetup.com/es/Data-on-Kub...) - DoK Days @ KubeCon We host hybrid events twice a year for people to get together and learn in-person or online. Details are published on our main website. (https://dok.community/) Connect With Us: https://dok.community/ https://www.youtube.com/c/DoKcommunit... https://twitter.com/DoKcommunity https://www.linkedin.com/company/6667... https://github.com/dokc