Apache Spark Runtimes in Microsoft Fabric – Runtime 1.3 based on Apache Spark 3.5

About this episode

Text from the YouTube description.

Microsoft Fabric Runtime is an Azure-integrated platform based on Apache Spark that enables the execution and management of data engineering and data science experiences. It combines key components from both internal and open-source sources, providing customers with a comprehensive solution. For simplicity, we refer to Microsoft Fabric Runtime powered by Apache Spark as Fabric Runtime.

Major components of Fabric Runtime:

Apache Spark - a powerful open-source distributed computing library that enables large-scale data processing and analytics tasks. Apache Spark provides a versatile and high-performance platform for data engineering and data science experiences.

Delta Lake - an open-source storage layer that brings ACID transactions and other data reliability features to Apache Spark. Integrated within Fabric Runtime, Delta Lake enhances data processing capabilities and ensures data consistency across multiple concurrent operations.

Default-level packages for Java/Scala, Python, and R - packages that support diverse programming languages and environments. These packages are automatically installed and configured, allowing developers to apply their preferred programming languages for data processing tasks.

Read more: https://learn.microsoft.com/en-us/fabric/data-engineering/runtime-1-3

🎙 Meet the Speakers:

👤 Guest Expert: Akshay Tayal, Senior Software Engineer | Fabric Product Engineering

LinkedIn: https://www.linkedin.com/in/akshaytayal/

👤 Host: Estera Kot, PhD, Principal Product Manager at Microsoft.

LinkedIn: https://www.linkedin.com/in/esterakot/

Twitter: https://twitter.com/estera_kot

👍 Liked this video? Don't forget to hit the 'Like' button and share it with your peers!