How to handle Data skewness in Apache Spark using Key Salting Technique

How to handle Data skewness in Apache Spark using Key Salting Technique

28.575 Lượt nghe
How to handle Data skewness in Apache Spark using Key Salting Technique
Handling the Data Skewness using Key Salting Technique. One of the biggest problem in parallel computational systems is data skewness. Data Skewness in Spark happens due to joining on a key that is not evenly distributed across the cluster, causing some partitions to be very large and not allowing Spark to process data in parallel. GitHub Link - https://github.com/gjeevanm/SparkDataSkewness Content By - Jeevan Madhur [LinkedIn - https://www.linkedin.com/in/jeevan-madhur-225a3a86] Editing By - Sivaraman Ravi [LinkedIn - https://www.linkedin.com/in/sivaraman-ravi-791838114/]