Step 8: Challenge
Apr 22 – May 13 | Challenge
Apr 22 – May 13 | Challenge
Welcome to the final step of the MEDomicsLab Testing Phase! We appreciate your engagement throughout this journey.
The objective here is that participants design their own model via the use of the Machine Learning (ML) Module. These models will be evaluated by the MEDomicsLab team using a fresh, private holdout set from the MIMIC database.
Before proceeding with Step 8 - Challenge of the MEDomicsLab Testing Phase, we recommend revisiting the documentation related to Step 6 - Create Model.
Download the new learning set provided in MEDomicsLab_TestingPhase_Step8.zip.
Create a model similarly to the one we created in Step 6 - Create Model using the Machine Learning (ML) Module. Follow these guidelines for creating your best possible model:
Train, optimize, and test models.
Compare different data combinations.
Choose a learning hypothesis and create a final model.
Once you have trained your best model, send it to sarah.denis@usherbrooke.ca with "Step 8 - Challenge | Submission" as the Subject of your email. You are allowed to send multiple submissions, but please note that we will only evaluate your latest submission.
For this ML Challenge, most of your work will therefore be done inside the Machine Learning (ML) Module. However, you are also welcome to use the other capabilities of the MEDomicsLab platform. For example, you may want to use the Exploratory Module to continue to gain useful insights about the dataset prior to the learning step.
However, to facilitate the Evaluation process of the models submitted by the participants, we please ask the following:
Only use the data we provided in the MEDomicsLab_TestingPhase_Step8.zip file.
Therefore, you should not be able to use the Extraction Module or the MEDprofiles package.
Do not change column names in the data we provided (this includes the use of the Groupping/Tagging tool).
Do not merge the time point CSV files together.
Dot not use the "Transform Columns tool".
Do not use the "PCA" utility of the "Feature Reduction tool".
If you use the "Spearman" utility of the "Feature Reduction tool", make sure to keep the subject_id and target columns in your dataset by enabling the "Merge unselected columns in the result dataset" and "Keep target in dataset" options.
Note that you new dataset will be placed in the reduced_features folder and that you will have to move it in your learning folder to retrieve it in the Learning Module.
If these instructions are not followed, we will not be able to evaluate your submission, and it will unfortunately be discarded.
Also, note that to create a model in the Learning Module, your CSV files must be placed under a "learning" folder and have the prefix "TX_" (e.g., T1_new_learning) as in the data we provided you.
Content
Intro 0:00
Rules 0:38
Dealing with Server Errors 4:03
Documentation 5:08
The Optimize node 8:58
General Advice 11:05
In Step 8 - Challenge, you will leverage the knowledge you have gained from the MEDomicsLab platform in the preceding steps of the Testing Phase. You are the master here !
If you wish, you can let us know in the email the name that we should use for the public ranking of your submission (e.g. MLrocks ). If not specified, we will use your full name.
Following this, we will make the ranking public and reveal the winners of both the ML Challenge and the Bug Finder Challenge at the wrap-up meeting scheduled for May 17. An invitation to this meeting will be sent out soon.