ML based Autotune for Apache Spark Jobs in MS Fabric performance optimization for recurrent jobs

In this episode of Fabric Espresso, we delve into Autotune. Estera Kot, along with Karuna Sagar Krishna and Yiwen Zhu unpack the technology, use cases and demo of Autotune – predictive optimization for Apache Spark Jobs - which automatically adjusts Apache Spark configuration to speed up workload execution and to optimize overall performance. Autotune configures three Apache Spark settings for each of your queries separately based on historical runs. Read more: https://learn.microsoft.com/en-us/fabric/data-engineering/autotune 🎙 Meet the Speakers: 👤 Guest Expert: Karuna Sagar Krishna | Principal Software Engineering Manager at Microsoft LinkedIn:https://www.linkedin.com/in/karuna-krishna-842211126/ 👤 Guest Expert: Yiwen Zhu | Principal Applied Data Scientist, Researcher at Microsoft LinkedIn: https://www.linkedin.com/in/yiwen-zhu-25139933/ 👤 Host: Estera Kot, PhD, Principal Product Manager at Microsoft. LinkedIn: https://www.linkedin.com/in/esterakot/ Twitter: https://twitter.com/estera_kot 👍 Liked this video? Don't forget to hit the 'Like' button and share it with your peers!