Using MLflow Tracking Server

AI/ML Accelerator - MLflow is specifically built to track your ML experiments on the DNAnexus platform environment via the ML JupyterLab (another app in the AI/ML Accelerator package) environment. A license is required in order to use the AI/ML Accelerator package. For more information, please contact DNAnexus Sales via [e-mail].

When To Use MLflow Tracking Server

After logging runs, experiments, and models to the DNAnexus storage by MLflow, you can open a Tracking Server GUI to organize and manage your ML experiments.

The MLflow Tracking Server GUI is a user-friendly interface designed for managing and visualizing machine learning experiments. It provides a centralized platform to log, track, and compare model parameters, metrics, and artifacts across different runs. The GUI enhances collaboration and reproducibility by offering clear insights into the lifecycle of your ML projects. For more information, please visit the MLflow documentation.

Open MLflow Tracking Server in ML JupyterLab

MLflow Tracking Server is a stand-alone HTTP server that serves multiple REST API endpoints for tracking runs/experiments stored on the DNAnexus project.

When an ML JupyterLab job is run, the MLflow Tracking Server will be automatically started in which the MLflow Backend and Artifact are saved in the DNAnexus project (where the ML JupiterLab job is launched).

To open the MLflow Tracking Server GUI, in the ML JupyterLab launcher, click on the MLflow symbol, and a new tab for MLflow Tracking Server will appear.

ML JupyterLab launcher with the integrated MLflow

Open MLflow Tracking Server as a DNAnexus Job

To open a Tracking Server, another option is to launch an MLflow job. You can do this with the UI or in the CLI

To Launch with UI

1. Find MLflow in your Tools Library

Simply open your Tools Library and search for "MLflow". If you cannot find it, you might have to obtain a license.

2: Start the job

You do not need to specify any input for the job, just simply click on 'Start Analysis' to launch the job.

3: Open your MLflow Tracking Server

Once your MLflow job is launched, you will be redirected to the Monitor screen. From there, click on the Open button.

Use the Open button in the Worker URL to use MLflow

Even when the Job State is "Running", it might take a few more minutes for the Platform to set up MLflow. When the job is not ready, you will see the below screen. In such cases, simply reload your browser after a few minutes.

The waiting screen of MLflow when the instance is not ready

To Launch with CLI

You can also launch your job with dxtoolkit:

dx run dxmlflow --name "My MLflow Tracking Server"

Resources

Full Documentation

To create a support ticket if there are technical issues:

1. Go to the Help header (same section where Projects and Tools are) inside the platform

2. Select "Contact Support"

3. Fill in the Subject and Message to submit a support ticket.