Microsoft Fabric Data Engineering - How to make the reference file work in Spark Job Definitions?
Episode 46 of Fabric Espresso, hosted by Dr. Estera Kot.
About this episode
Text from the YouTube description.
Join us in this Fabric Espresso episode as Qixiao Wang and Estera Kot explore the integration of reference files in Spark Job Definitions in Microsoft Fabric.
🎙 Meet the Speakers:
👤 Guest Expert: Qixiao Wang, Principal Program Manager in Microsoft
Linkedin: https://www.linkedin.com/in/%E5%90%AF%E9%9C%84-%E7%8E%8B-24368233/
👤 Host: Estera Kot, Principal Product Manager at Microsoft.
LinkedIn: https://www.linkedin.com/in/esterakot/
Twitter: https://twitter.com/estera_kot
👍 Liked this video? Don't forget to hit the 'Like' button and share it with your peers!
🔔 Stay Updated: Subscribe to our channel for the latest in Microsoft Fabric Data Engineering, Data Science, and groundbreaking tech. Hit the notification bell to never miss out on the Fabric Espresso series!
Chapters
- 0:00 Introduction
- 0:06 Spark Job Definition Overview
- 1:05 Reference Files and Modules
- 2:55 Demo: Creating a Spark Job
- 4:00 Code Structure and Execution
- 5:15 Upload Options
- 6:18 Git Integration Support
- 7:27 Java/Scala and Feature Updates
- 9:00 Job Services and Roadmap
- 10:45 User Feedback and Futures
- 12:20 Spark SQL and Closing