Hyperparameter tuning in Fabric Data Science

About this episode

Text from the YouTube description.

Hyperparameter tuning involves determining the optimal values for parameters not learned by the machine learning model during training but set by the user prior to the initiation of the training process. These parameters, known as hyperparameters, include the learning rate, the number of hidden layers in a neural network, the regularization strength, and the batch size. The performance of a machine learning model can be highly sensitive to the selection of hyperparameters, with the optimal set varying significantly based on the specific problem and dataset. Consequently, hyperparameter tuning is a crucial step in the machine learning pipeline, substantially influencing the model's accuracy and generalization performance.

In the context of Fabric, data scientists can utilize FLAML, a lightweight Python library designed for the efficient automation of machine learning and AI operations, to meet their hyperparameter tuning needs. Within Fabric notebooks, users can employ `flaml.tune` for cost-effective hyperparameter tuning.

https://learn.microsoft.com/en-us/fabric/data-science/hyperparameter-tuning-fabric

https://blog.fabric.microsoft.com/en-US/blog/introducing-code-first-automl-and-hyperparameter-tuning-now-in-public-preview-for-fabric-data-science/

🎙 Meet the Speakers:

👤 Guest from Microsoft Fabric Product Group: Misha Desai, Senior Program Manager at Microsoft

Misha is a Senior Product Manager based in Seattle, WA, specializing in model tracking, training, and governance within the Fabric Data Science team.

Linkedin: www.linkedin.com/in/misha-desai-6034a362

👤 Host: Estera Kot, PhD, Principal Product Manager at Microsoft.

LinkedIn: https://www.linkedin.com/in/esterakot/

Twitter: https://twitter.com/estera_kot

👍 Liked this video? Don't forget to hit the 'Like' button and share it with your peers!