22 Optimize Joins in Spark & Understand Bucketing for Faster joins |Sort Merge Join |Broad Cast Join

Video explains - How to Optimize joins in Spark ? What is SortMerge Join? What is ShuffleHash Join? What is BroadCast Joins? What is bucketing and how to use it for better performance? Chapters 00:00 - Introduction 00:48 - How Spark Joins Data ? 03:25 - Shuffle Hash Join 04:20 - Sort Merge Join 04:59 - Broad Cast Join 07:50 - Optimize Big and Small Table Join 13:32 - Optimize Big and Big Table Join 16:09 - What is Bucket in Spark ? 18:39 - Optimize Join with Buckets Local PySpark Jupyter Lab setup - https://youtu.be/WhxljT3IfdM Python Basics - https://www.learnpython.org/ GitHub URL for code - https://github.com/subhamkharwal/pyspark-zero-to-hero/blob/master/18_optimizing_joins.ipynb The series provides a step-by-step guide to learning PySpark, a popular open-source distributed computing framework that is used for big data processing. New video in every 3 days ❤️ #spark #pyspark #python #dataengineering