How to Read Spark DAGs | Rock the JVM

Written form to keep for later: https://blog.rockthejvm.com/spark-dags/ Check out the Spark performance courses: https://rockthejvm.com/p/spark-optimization https://rockthejvm.com/p/spark-performance-tuning Download Spark: https://spark.apache.org/downloads.html Related video on reading Spark Query Plans: https://youtu.be/UZt_tqx4sII This video is for Spark programmers who know the essentials (e.g. create a DataFrame, basic selects/joins) and wants a sneak peek at how Spark works internally and get an essential skill for performance analysis and improvement. In this video I'll show you how Spark creates computation dependencies before it can run anything, and I'll show you how to read and interpret Directed Acyclic Graphs (DAGs) and identify performance bottlenecks while the job is running. Follow Rock the JVM on: LinkedIn: https://linkedin.com/company/rockthejvm Twitter: https://twitter.com/rockthejvm Blog: https://rockthejvm.com/blog ------------------------------------------------------------------------- Home: https://rockthejvm.com -------------------------------------------------------------------------