Introduction to Data Quality Manager

Purpose of This Guide

This guide provides step-by-step instructions and conceptual explanations to help you use Data Quality Manager to profile, remediate, and monitor data quality effectively. It is designed to help you understand the core features, workflows, and best practices for managing data quality in clinical and medical research environments.

What is Data Quality Manager

Data Quality Manager is an application on the DNAnexus platform that enables you to systematically assess data quality, apply rule-based remediation, and track data quality over time.

The application supports end-to-end data quality management by providing:

• Data Profiler: to profile data and identify quality issues

• Data Refinery: to apply governed, rule-based remediation to clean, transform, and harmonize data

• Data Monitor: to monitor data quality and remediation performance across runs

Who This Guide Is For

This guide is intended for you if you are involved in managing, preparing, or overseeing clinical and medical research data, including:

• Clinical Data Managers

• Directors of Clinical Informatics

• Data Engineers and Data Scientists

• Research and Analytics Teams

You do not need advanced programming skills to use Data Quality Manager; however, familiarity with clinical data and basic data quality concepts will help you get the most value from the application.

Key Concepts and Terminology

The following key concepts are used throughout this guide:

• Data profiling: The process of analyzing your datasets to understand their structure, content, and data quality characteristics, such as completeness, validity, and uniqueness.

• Data remediation: The application of defined rules to clean, transform, and harmonize your data to resolve identified quality issues.

• Remediation rule / business rule: A rule that defines a Condition and an Action describing how specific data issues should be addressed.

• Run: A single execution of data remediation on one dataset.

• Data quality assessment report: A summary report that presents quantitative metrics used to evaluate data quality and remediation outcomes, such as missing value rates, duplicate counts, and completeness levels.

• Rule violation: A situation occurring when a data record satisfies the condition of a remediation rule, triggering the rule’s action.