Data Engineering Interview - Netflix Clickstream Data Pipeline

Join the waitlist for Exponent’s Data Engineering Interview Course: https://bit.ly/4cmpq34 In this video, an expert in data engineering discusses building a near real-time data ingestion pipeline for Netflix clickstream and playback data. The focus is on key metrics like customer churn and path analysis, with insights into using technologies like Kafka, Spark, and NoSQL databases. The video also covers the importance of scalable, time-sensitive data processing and touches on trade-offs in data storage and system design. Want to practice peer-to-peer mock interviews? https://bit.ly/3Xmj8wq Chapters - 00:00 - Intro 01:15 - Data pipeline for Netflix metrics monitoring 05:52 - Path analysis and playback insights 09:42 - Product insights and pipeline design overview 12:31 - Netflix’s user distribution and metrics 16:51 - Netflix’s scalability: Up to 50,000 users per second 25:13 - Spark streaming, Kafka, Flink, data lake 28:08 - Kafka vs Data lakes for analytics 30:16 - NoSQL database performance 37:41 - Big data pipeline design principles Watch data science mock interviews from Exponent: - Probability, P-value and Confidence Intercals: https://youtu.be/-daFO-i-SFY - Retry Transaction ft. Paypal Data Scientist: https://youtu.be/uCiBkaA4V6M - Amazon Data Science Interview: Linear Regression:https://youtu.be/PYZPmpnbUxo - Snap Data Science Mock Interview: Improve Camera Speed: https://youtu.be/6bAyvNE2CeM 👉 Subscribe to our channel: http://bit.ly/exponentyt 🕊️ Follow us on Twitter: http://bit.ly/exptweet 💙 Like us on Facebook for special discounts: http://bit.ly/exponentfb 📷 Check us out on Instagram: http://bit.ly/exponentig 📹 Watch us on TikTok: https://bit.ly/exponenttiktok ABOUT US: Did you enjoy this interview question and answer? Want to land your dream career? Exponent is an online community, course, and coaching platform to help you ace your upcoming interview. Exponent has helped people land their dream careers at companies like Google, Microsoft, Amazon, and high-growth startups. Exponent is currently licensed by Stanford, Yale, UW, and others. Our courses include interview lessons, questions, and complete answers with video walkthroughs. Access hours of real interview videos, where we analyze what went right or wrong, and our 1000+ community of expert coaches and industry professionals, to help you get your dream job and more!