Advancing Spark - Getting Started with Ganglia in Databricks

As a first video back for 2022, we thought we'd take a look back at one of the most useful, but overlooked tools within a Databricks Administrator's toolbelt. We hear from many people in the community that they're having trouble monitoring their clusters, figuring out how utilised they are and diagnosing performance problems. Ganglia is an incredibly useful (but initially intimidating) tool that's baked in to the Databricks workspace! In this video Simon walks through how he uses Ganglia when looking at a specific load problem, what you can do, what you can't do, and gives you what you need to get started monitoring your cluster performance We've dug into the Spark UI previously, so if you're just getting started, check it out here: https://youtu.be/rNpzrkB5KQQ As always, get in touch if Advancing Analytics can help you on your analytics journey