Introduction to MLflow

AI/ML Accelerator - MLflow is specifically built to track your ML experiments on the DNAnexus platform environment via the ML JupyterLab (another app in the AI/ML Accelerator package) environment. A license is required in order to use the AI/ML Accelerator package. For more information, please contact DNAnexus Sales via [e-mail].

What is AI/ML Accelerator - MLflow?

AI/ML Accelerator - MLflow is a custom version of MLflow on the DNAnexus platform. MLflow is designed to manage the entire lifecycle of machine learning (ML) development. It is widely used to track experiments, package ML code into reproducible formats, deploy models, and manage their lifecycle in production. With this custom version on the DNAnexus platform, you are able to easily manage your ML project and leverage the collaboration on the DNAnexus platform.

Why use the AI/ML Accelerator - MLflow?

Using MLflow on the DNAnexus platform combines powerful ML lifecycle management with the robust data protection platform. It ensures sensitive metadata and artifacts remain within your control, supporting compliance with regulations like GDPR and HIPAA. This setup enhances collaboration, eliminates reliance on third-party providers, and scales with your growing ML operations, all while ensuring end-to-end security.

Core features of AI/ML Accelerator - MLflow

1. Ability to Track and Store Model Experiments in DNAnexus Platform:

• AI/ML Accelerator - MLflow enables seamless tracking and storage of model experiments directly in DNAnexus platform storage, leveraging the secure and scalable infrastructure.

• Your experiment metadata, logs, and artifacts will be stored in a secured environment with role-based access control. The synchronization mechanisms allow seamless data integration between the ML development environments and DNAnexus platform.

2. Ability to Query the Runs

• The querying interface allows users to filter, sort, and compare runs based on parameters, metrics, and tags.

• The system allows you to search your MLflow runs through both MLflow GUI and Python API. For detailed instructions, please refer to the MLflow documentation.

3. Ability to Log and Load the Models

• AI/ML Accelerator - MLflow supports logging of models, custom artifacts, and associated metadata directly to the DNAnexus platform, ensuring compatibility across environments.

• You are able to dynamically load models from the DNAnexus platform with version control and dependency management. Custom formats are supported for logging and retrieval tailored to your workflows.

4. Ability to Register the Model Versions

• The model registry provides centralized management for tracking model versions and lifecycle stages (e.g., staging, production).

• The system allows you to compare versions, perform rollbacks, and enrich registered models with custom metadata.

5. Collaboration Space for Sharing Runs, Experiments, and Models

• The shared workspace enables users to share runs, experiments, and models with collaborators on the DNAnexus project.

• The role-based permissions ensure that only authorized users can view or modify shared resources.

Resources

Full Documentation

To create a support ticket if there are technical issues:

1. Go to the Help header (same section where Projects and Tools are) inside the platform

2. Select "Contact Support"

3. Fill in the Subject and Message to submit a support ticket.