Hyperparameters optimization

In this tutorial, we will learn how to use MEDfl to optimize the hyperparameters of the model and apply them effectively.

Introduction

We will explore how to use MEDfl to optimize the hyperparameters of your machine-learning models. Hyperparameter optimization is a crucial step in enhancing the performance of your models. MEDfl provides robust tools to streamline this process, allowing you to fine-tune your models efficiently. By the end of this tutorial, you will understand how to leverage MEDfl for hyperparameter optimization and utilize the optimized parameters to improve your model's accuracy and effectiveness.

Let's get started!

Video tutorial

Optimization methods

MEDfl gives the possibility to optimize hyperparameters with 3 different methods

• Grid Search Optimization: A straightforward method where the user specifies a list of metrics to optimize, such as the number of layers, hidden size, and others.

• Optuna Central Optimization: Utilizes Optuna to optimize parameters on the central server. Users can specify Optuna parameters like metric, direction, optimization algorithm, and intervals for each hyperparameter.

• Optuna Federated Optimization: Uses Optuna for hyperparameter optimization in a federated manner. Optimization occurs during the execution of the federated pipeline, adapting parameters based on distributed data.

Run optimization

To run the optimization click on the run button and a new popup will show to verify the optimization configuration then click the button "Optimize hyperparameters"

After running the optimization you will see a progress bar showing the progress of the execution

Optimization results

Once the execution is completed a popup will show the results

• GRID SEARCH results : The results contains the best score and best hyperparameters

• Optuna Results: The results contain the best score and best hyperparameters and some statistical graphs

Save Optimization results

MEDfl allows users to save the optimization results into files and visualize them on separate screens. This feature helps in easily tracking and analyzing the performance of different hyperparameter configurations.

to save the results click on the button "Save results" and an input will show to give the file name

The application will save two files under the specified name (JSON and medflopt) in the "optimization" folder.

EXPERIMENTS                          
    ├───FL                 
        └─── Optimization
             └─── optimization_results.json       -> json file 
             └─── optimization_results.medflopt   -> medflres file

You can visualize the file by double-clicking on them directly