High Concurrency Mode in Microsoft Fabric

About this episode

Text from the YouTube description.

Fabric Espresso Presents: High Concurrency Mode in Microsoft Fabric. Learn about how high concurrency mode works in Fabric Spark for Notebooks 🔥🔥🔥

📌 Ever wondered how to optimize your Spark sessions in Microsoft Fabric for both Data Engineering and Data Science workloads? This video is your one-stop guide to understanding High Concurrency Mode in Fabric Spark for Notebooks. Learn how this mode allows for the independent execution of multiple items within a single Spark session, providing you with an instant run experience!

📚 What You'll Learn:

✅ High Concurrency Mode: What it is and how it allows users to share the same Spark sessions across multiple notebooks.

✅ Session Sharing Conditions: Key conditions for sharing sessions like user boundaries and lakehouse configurations.

✅ Performance Boost: Discover how Custom Pools with High Concurrency Mode offer a 36X faster session start compared to standard Spark sessions.

Read more here: https://learn.microsoft.com/en-us/fabric/data-engineering/high-concurrency-overview

🎙 Meet the Speakers:

👤 Guest from Product Group: Santhosh Kumar Ravindran, Senior Product Manager

Santhosh Ravindran currently leads Spark Compute and Settings for Microsoft Fabric Spark. He focuses on building capabilities that meet the data engineering needs like Spark Pools, Queueing and Scheduling, and Job orchestration for big data workloads in large enterprises using Spark. Prior to this, Santhosh was a Product Manager and Engineering Lead building Metadata scanners, access policy orchestration, lineage and data catalog systems for enterprise data estates as part of the Microsoft Governance Platform (Microsoft Purview).

Linkedin: https://www.linkedin.com/in/thisissanthoshkumar/

Twitter: https://www.twitter.com/iamsanthoshkr

👤 Host: Estera Kot: Senior Product Manager at Microsoft
LinkedIn: https://www.linkedin.com/in/esterakot/
Twitter: https://twitter.com/estera_kot