Understanding Databricks & Apache Spark Performance Tuning: Lesson 02 - Spark Hardware

Following up on Databricks Performance Tuning with the best place to start: allocating Spark clusters. If you don't allocate sufficient resources, nothing else will fix the problem. How many nodes? How large should the driver and workers be? Do you need GPUs or CPUs? Should you use Photon? These and many more questions will be covered in detail. Support me on Patreon https://www.patreon.com/bePatron?u=63260756 Slides https://github.com/bcafferky/shared/blob/master/DatabricksPerfTuning/lesson02_DatabricksPerfTuningHardware.pdf Referenced Video by Daniel Tomes, Databricks https://www.youtube.com/watch?v=daXEp4HmS-E&t=2111s See the Full Playlist Here: https://www.youtube.com/playlist?list=PL7_h0bRfL52qw5SkSI_P_kURU2njM1UO-