Microsoft Fabric Spark Diagnostic Emitter for Logs and Metrics

About this episode

Text from the YouTube description.

Fabric Apache Spark Diagnostic Emitter for Logs and Metrics is now in public preview. This new feature allows Apache Spark users to collect Spark logs, job events, and metrics from their Spark applications and send them to various destinations, including Azure Event Hubs, Azure Storage, and Azure Log Analytics. It provides robust support for monitoring and troubleshooting Spark applications, enhancing your visibility into application performance.

What Does the Diagnostic Emitter Do?

The Fabric Apache Spark Diagnostic Emitter enables Apache Spark applications to emit critical logs and metrics that can be used for real-time monitoring, analysis, and troubleshooting. Whether you’re sending logs to Azure Event Hubs, Azure Storage, or Azure Log Analytics, this emitter simplifies the process, allowing you to collect data seamlessly and store it in your preferred destinations.

Blog post: https://blog.fabric.microsoft.com/en-US/blog/announcing-the-fabric-apache-spark-diagnostic-emitter-collect-logs-and-metrics/

🎙 Meet the Speakers:

👤 Guest Expert: Jenny Jiang, Principal Product Manager | Fabric Data Engineering

LinkedIn: https://www.linkedin.com/in/jenny-jiang-8b57036/

👤 Host: Estera Kot, PhD, Principal Product Manager at Microsoft.

LinkedIn: https://www.linkedin.com/in/esterakot/

Twitter: https://twitter.com/estera_kot

👍 Liked this video? Don't forget to hit the 'Like' button and share it with your peers!