ML based Autotune for Apache Spark Jobs in MS Fabric performance optimization for recurrent jobs
Episode 65 of Fabric Espresso, hosted by Dr. Estera Kot.
About this episode
Text from the YouTube description.
In this episode of Fabric Espresso, we delve into Autotune. Estera Kot, along with Karuna Sagar Krishna and Yiwen Zhu unpack the technology, use cases and demo of Autotune – predictive optimization for Apache Spark Jobs - which automatically adjusts Apache Spark configuration to speed up workload execution and to optimize overall performance. Autotune configures three Apache Spark settings for each of your queries separately based on historical runs.
Read more: https://learn.microsoft.com/en-us/fabric/data-engineering/autotune
🎙 Meet the Speakers:
👤 Guest Expert: Karuna Sagar Krishna | Principal Software Engineering Manager at Microsoft
LinkedIn:https://www.linkedin.com/in/karuna-krishna-842211126/
👤 Guest Expert: Yiwen Zhu | Principal Applied Data Scientist, Researcher at Microsoft
LinkedIn: https://www.linkedin.com/in/yiwen-zhu-25139933/
👤 Host: Estera Kot, PhD, Principal Product Manager at Microsoft.
LinkedIn: https://www.linkedin.com/in/esterakot/
Twitter: https://twitter.com/estera_kot
👍 Liked this video? Don't forget to hit the 'Like' button and share it with your peers!