Databricks Cost Monitoring With System Tables and Dashboards

This video explores system tables in Databricks for governance purposes. I explain different types of system tables and relationships between these tables and demonstrate how they can be enabled for usage. I also explained how to create cost and performance analysis dashboards for intelligent monitoring. Chapters: 00:00- Introduction and agenda 02:09- Overview of System Tables 08:06- Demo: Enabling System Tables 13:59- System tables-deep-dive 25:06- Using Databricks Dashboards to monitor Databricks events 26:57- Demo- creating dashboards 29:35- Cost and performance analysis dashboard Please subscribe: https://www.youtube.com/channel/UC8d958MxE2t1dr27QNqoOhA The demo notebook is available here: https://github.com/fazizov/youtube/blob/main/Data%20engineering%20with%20Databricks/System%20Tables/System%20tables.dbc You can read more about system tables here: https://docs.databricks.com/aws/en/admin/system-tables/