> For the complete documentation index, see [llms.txt](https://academy.dnanexus.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://academy.dnanexus.com/data-quality-manager/getting-started-with-data-quality-manager.md).

# Getting Started with Data Quality Manager

A license is required to access the Data Quality Manager on the DNAnexus Platform. For more information, please contact DNAnexus Sales (via <sales@dnanexus.com>).

### Accessing Data Quality Manager on DNAnexus Platform

There are two ways to run the application:

1. Direct Access: Go to this [link](https://platform.dnanexus.com/panx/tool/app/data_quality_manager) to open the app and launch a job.
2. Platform Navigation: Click on the top navigation bar, then select Open Tools > search for the Data Quality Manager app, and run the app.

The app requires no input to launch a job, input files can be specified in the job once it’s ready.

### Open the Data Quality Manager UI

After launching a job, you will be redirected to the Monitor screen. From there, click the job name to view the job details.

It may take a few minutes for the job to be ready. The Data Quality Manager is an [HTTPS application](https://documentation.dnanexus.com/developer/apps/https-applications) on the DNAnexus Platform, so it is accessed via the Job URL. Once the job is ready, click Open Worker URL to open the job UI.&#x20;

If you attempt to click the button before the job is ready, you may encounter a "502 Bad Gateway" error. This is not a problem, it simply means you need to wait a bit longer before the environment is fully prepared.

### Access the Home Page

After clicking the Open URL button to open the Data Quality Manager job, you will first arrive at the Home Page, which serves as the central entry point to all core components of the application.

<img src="https://1979569080-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPtCOm9rXoRi4P9rh1ET8%2Fuploads%2FMOwUcdrgsiBNPMu6Ss6V%2Funknown.png?alt=media&amp;token=cf1fb4d7-7f77-416c-8022-ded42d029da3" alt="" height="385" width="444">

Data Quality Manager homepage

From the Home Page, you can access:

* Data Profiler to inspect data structure, review quality metrics, and identify data issues of all datasets that have been profiled in the project.
* Data Refinery to define remediation rules and apply them to your data.
* Data Monitor to review data quality trends and remediation history.

### High-Level Recommended Workflow

<img src="https://1979569080-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPtCOm9rXoRi4P9rh1ET8%2Fuploads%2F7xmoi1Bl585ndDAf4aFD%2Funknown.png?alt=media&amp;token=d8102e44-3825-4696-a6f2-5c1cf89559a7" alt="" height="173" width="576">

Data Quality Manager high-level workflow

### Input Specification

To use Data Profiler, you need to provide a dataset name and select the required Table data. You can also optionally include a Data dictionary, Entity dictionary, and Coding data (similar to those needed for Apollo data ingestion with the Data Model Loader). See the table below for more details.

<img src="https://1979569080-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPtCOm9rXoRi4P9rh1ET8%2Fuploads%2Fjx7Po3EZYZ9sXwtqslOj%2Funknown.png?alt=media&amp;token=b336ddeb-82a3-4506-8c5e-1d243a929f03" alt="" height="416" width="390">

You can provide input data by selecting either individual files or an entire folder

| Input name        | <p>Required/</p><p>Optional</p> | Input format supported                               | Description                                                                                                                                                                                                                                                                                                                                                             |
| ----------------- | ------------------------------- | ---------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Dataset name      | Required                        | N/A                                                  | Name of your dataset.                                                                                                                                                                                                                                                                                                                                                   |
| Table data        | Required                        | <p>CSV files</p><p>TSV files</p><p>Parquet files</p> | <p>This is the data that will be profiled by this application. Each file is a table in your dataset.</p><p>You can provide input data by selecting individual files or an entire folder. When selecting a folder, all supported files within that folder are included in the dataset.</p>                                                                               |
| Data dictionary   | Optional                        | A CSV file                                           | <p>The Data Dictionary file of the dataset. This file follows the format of DNAnexus Data Model Loader dictionary (<a href="https://documentation.dnanexus.com/developer/ingesting-data/data-model-loader/data-file-inputs-data-model-loader#data-dictionary">Details</a>).</p><p>If not provided, the relationship between tables in the dataset will be inferred.</p> |
| Entity dictionary | Optional                        | A CSV file                                           | The dictionary describes tables/entities that the data files represent ([Details](https://documentation.dnanexus.com/developer/ingesting-data/data-model-loader/data-file-inputs-data-model-loader#entity-dictionary)).                                                                                                                                                 |
| Coding data       | Optional                        | A CSV file                                           | This file must contain the coding\_name header to link categorical fields. These names must match the corresponding values in the Data Dictionary ([Details](https://documentation.dnanexus.com/developer/ingesting-data/data-model-loader/data-file-inputs-data-model-loader#codings)).                                                                                |

### Working with Multiple Datasets

You can work with multiple datasets within a single job, or access datasets that were profiled in previous jobs within the same project.

Each dataset is processed independently, allowing you to:

* Profile data quality for each dataset separately
* Apply different remediation rules or rule sets per dataset
* Manage and review datasets across runs in a consistent workflow

<img src="https://1979569080-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPtCOm9rXoRi4P9rh1ET8%2Fuploads%2F0vxfLXof93HbDx9g0WsT%2Funknown.png?alt=media&amp;token=fe76f71f-cd4c-44d9-a6a3-39b5b3272bff" alt="" height="329" width="454">

You can profile multiple datasets and access profile of other&#x20;

### Advanced Usage: Non-Interactive / Batch Execution

Data Quality Manager also supports a non-interactive execution mode through the DNAnexus CLI. This allows you to launch remediation jobs programmatically without accessing the interactive UI, making it suitable for batch processing, workflow automation, and large-scale operational runs.

In non-interactive mode, you can:

* Specify input datasets directly from the command line
* Apply an existing Rule Set automatically
* Launch multiple remediation jobs in batch workflows
* Integrate Data Quality Manager into automated pipelines

#### Example Command

```
dx run app-data_quality_manager \
-itable_files="/data/remediation_test/lab_results.csv" \
-itable_files="/data/remediation_test/diagnoses.csv" \
-itable_files="/data/remediation_test/medications.csv" \
-itable_files="/data/remediation_test/patients.csv" \
-itable_files="/data/remediation_test/visits.csv" \
-idataset_name="Test Non-interactive" \
-irule_set_id="xxxxxxxxx" \
--name "Non-interactive Mode" -y
```

#### Parameters

* `table_files:` Path (or ID) of the input table to include in the dataset
* `dataset_name:` Name of the dataset created for the remediation run
* `rule_set_id:` Identifier of the Rule Set to apply during remediation
* `--name:` Name of the DNAnexus job
* `-y:` Automatically confirms job submission

This execution mode is recommended for repeated remediation workflows, scheduled processing, and production-scale operations.