nf-core/scdownstream

Necessary Disclaimers and Legal

The user is responsible for reviewing and complying with the license requirements of the software, and data referenced in this documentation.

Users are responsible for the costs associated with analyzing data, software and its storage in their project spaces.

Instance type availability and pricing are subject to the contract between the user or the user’s organization and DNAnexus.

Citations for nf-core/scdownstream

nf-core is a community-driven collection of curated bioinformatics pipelines built with Nextflow, providing standardized, scalable, and portable workflows for reproducible analysis across diverse computing environments. Developed by the nf-core community, nf-core/scdownstream supports quality control, data integration, and presentation-ready outputs for single-cell downstream analysis. If you use nf-core/scdownstream, please cite the pipeline using nf-core/scdownstream. Additional references for integrated tools are listed in the pipeline’s CITATIONS.md.

The following datasets were used in this documentation as example runs:

• GSE147424: nf-core/scdownstream test dataset.

• GSE171555: Yu C et al. Med (N Y). 2021;2(6):755–772.e5. PMID: 33870241

• GSE174609: Lee H et al. J Transl Med. 2022;20(1):504. PMID: 36329504

Overview

nf-core/scdownstream is a bioinformatics pipeline for processing already-quantified single-cell RNA-seq data. It takes a samplesheet (.csv) as input and performs quality control, integration, dimensionality reduction and clustering. It produces an integrated .h5ad file and an extensive quality control report.

Since the nf-core/scdownstream pipeline is still under development and does not yet have a tagged release, we pulled the stable commit (753add5, March 2026) and built the applet. Please refer to the official document and Github for further details.

Where to access nf-core/scdownstream

The nf-core/downstream folder on DNAnexus is organized as follows:

scdownstream/
├── datasets/             # Input datasets for the pipeline
├── example_results/      # Example output results from a completed run
├── notebooks/            # Notebooks for sample sheet preparation
├── singleR/              # SingleR reference datasets for cell type annotation
├── soft_configs/         # Resources & tool configuration files
└── scdownstream_753add5  # Nextflow applet

1. Applet: scdownstream_753add5: Built from commit 753add5 (March 2026).

2. Datasets: Three GEO datasets are provided, each containing the input data and its corresponding sample sheet:

datasets/
├── GSE147424_test_dataset/
├── GSE171555_RNA/
├── GSE174609_scrnaseq_output/
├── GSE147424_samples.csv       # Sample sheet for GSE147424
├── GSE171555_samples.csv       # Sample sheet for GSE171555
├── GSE174609_samples.csv       # Sample sheet for GSE174609
└── GSE174609_SraRunTable.csv   # SRA metadata table 

See the Example datasets section below for further details.

1. Notebooks

Three notebooks are provided: two for sample sheet preparation and one for exploring results.

notebooks/
├── GSE171555_prepare_input_*.ipynb    # Sample sheet preparation (GSE171555)
├── GSE174609_prepare_input_*.ipynb    # Sample sheet preparation (GSE174609)
└── GSE174609_*_result_explore.ipynb   # Result exploration (GSE174609)

1. SingleR reference datasets

Pre-downloaded celldex reference datasets and an example .csv file are provided. See the SingleR reference section below for further information.

singleR/
├── celldex_hpca__2024-02-26_h5_se.tar         # HPCA reference
├── celldex_monaco_immune__2024-02-26_h5_se.tar # Monaco Immune reference
└── celldex_references_example.csv      

1. Soft configuration files

Configuration files are provided as examples for users to adapt for their own runs. They are organized in two groups:

• Dataset-specific GPU runs:

• soft_config_GSE171555_gpu.config: GPU run for GSE171555 (5 samples).

• soft_config_GSE174609_gpu_qsize.config: GPU run for GSE174609 (12 samples).

• Test profiles:

• soft_config_test.config: Minimal test profile (CPU only) for a fast pipeline check.

• soft_config_test_full.config: Full test profile with GPU + CPU setup and full tool selection for a complete pipeline check.

1. Example results

Example output from completed pipeline runs are provided for reference, corresponding to each configuration profile:

example_results/
├── test_profile/       # Output from soft_config_test.config
├── test_full_profile/  # Output from soft_config_test_full.config
├── test_GSE171555/     # Output from soft_config_GSE171555_gpu.config
└── test_GSE174609/     # Output from soft_config_GSE174609_gpu_qsize.config

Copying applet and data into a project

To use the datasets and applet, first copy them from the public project into your own project:

1. Create a new project billed to your organization. See Setting Up a Project for instructions.

2. Go to the Resources tab and find the project "Public Datasets AWS US East". Navigate to scdownstream folder.

3. Select the folders and files you need (e.g., datasets/, singleR/, soft_configs/, notebooks/, and the applet scdownstream_753add5).

4. Click Copy from the top right menu and select the project you created in Step 1.

5. Go to your project to start running the pipeline.

Prepare your input

Samplesheet input

The pipeline requires a .csv sample sheet with the following format. Here is the example of minimal samplesheet:

sample,filtered,unfiltered,batch_col
SRR28679756,/path/to/SRR28679756_filtered_matrix.h5ad,/path/to/SRR28679756_raw_matrix.h5ad,sample
SRR28679757,/path/to/SRR28679757_filtered_matrix.sce.rds,/path/to/SRR28679757_raw_matrix.sce.rds,sample
SRR28679758,/path/to/SRR28679758_filtered_matrix.seurat.rds,/path/to/SRR28679758_raw_matrix.seurat.rds,sample

Please refer to GSE147424_samples.csv in the folder: datasets/ on the platform for a complete example.

Note: Remember to update the file paths in the example sample sheet to point to your actual data location on the platform before running the pipeline.

To simplify sample sheet preparation, a template notebook was prepared. The script scans the sample directory for .h5ad, .h5, and .rds files, extracts the sample ID from each filename (prefix before the first underscore), and includes the full DNAnexus file path. Please check our example sample sheet file: GSE171555_samples.csv and example notebook: GSE171555_prepare_input_scdownstream.ipynb on the platform.

For additional optional columns, refer to the nf-core/scdownstream user guide.

SingleR reference in the pipeline

You can provide a .csv file with information about the celldex references to use for the singleR cell type annotation with the celldex_reference parameter. The existing references are described in the celldex package description. SingleR provides several reference datasets including blueprint_encode, dice, hpca, immgen, monaco_immune, mouse_rnaseq, and novershtern_hematopoietic (see full list at https://bioconductor.org/packages/release/data/experiment/html/celldex.html ).

Here, we downloaded two references: HPCA (Human Primary Cell Atlas) for broad cell type annotation using label.main, and Monaco Immune for detailed immune cell subtype annotation using label.fine (version 2024-02-26).

celldex_references_example.csv
id,label,reference,version
hpca_celldex,label.main,hpca,2024-02-26
immune_celldex,label.fine,monaco_immune,2024-02-26
hpca_direct,label.main,/path/to/celldex_hpca__2024-02-26_h5_se.tar,
immune_direct,label.fine,/path/to/celldex_monaco_immune__2024-02-26_h5_se.tar,

Please refer to celldex_references_example.csv in the singleR/ folder on the platform for the complete file.

Example datasets

Please find all datasets and sample sheets in the datasets/ folder on the platform.

Sample files from nf-core/scdownstream

Downloaded examples from the test dataset of nf-core/scdownstream (Github), created a samplesheet: GSE147424_samples.csv.

GSE171555

We prepared an example of 5 samples from Single-cell RNA-seq of a human PBMC COVID-19 dataset - GSE171555. We provide a notebook here to show how to prepare this sample sheet:

project-xxx:/scdownstream/notebooks/GSE171555_prepare_input_scdownstream.ipynb

Output from nf-core/scrnaseq pipeline (GSE174609)

Downloaded PBMC scRNA-seq data from paired pre-/post-treatment periodontitis patients and healthy donors (GSE174609) for real data testing:

• 12 samples total, with good expected cell counts (~10,000).

• Existing SingleR labels in the publication will serve as a reference for nf-core/scdownstream.

• Multiple batches and conditions make this dataset suitable for testing integration methods in nf-core/scdownstream.

Run the pipeline

Minimal test

Use the minimal test profile to quickly validate the pipeline. We created soft_config_test.config with adjusted resource limits:

process {
    resourceLimits = [
        cpus: 16,
        memory: '60.GB',
        time: '24.h'
    ]
}

Run via UI

1. Click on the applet and Run the analysis.

1. Provide Job Name and choose Output destination.

1. Click the input file box to select a sample sheet (.csv) file.

1. Select GSE147424_samples.csv in the datasets folder for the minimal test and click Select.

1. Go to celldex_reference box and click the box to select the celldex reference file.

1. Choose celldex_references_example.csv in folder singleR, then click 'Select'.

Note: Before selecting celldex_references_example.csv, ensure that the file paths inside it have been updated to match your project's storage location.

1. Go to the Soft Configuration File box to select soft_config_test.config in folder soft_configs.

1. Under the COMMON section: enter the output directory name (e.g., test_profile). The pipeline will create a folder with this name inside the destination folder you select to save all output files.

1. Select True to save intermediate files or False to skip them.

2. Under the NEXTFLOW OPTIONS section, enter -profile docker in Nextflow Run Options box.

1. Once all inputs are filled in, click Start Analysis in the top right corner to launch the pipeline.

2. Go to the MONITOR tab to check the pipeline execution progress. The Inputs and Outputs section at the bottom summarizes all input parameters and the published output files from the run.

Run via command line

Alternatively, you can run the pipeline via the command line using the DNAnexus CLI (dx). Replace project-xxx with your DNAnexus project ID or Public Dataset project.

dx run project-xxx:/scdownstream/scdownstream_753add5 \
  -ioutdir='test_profile' \
  -iinput='project-xxx:/scdownstream/datasets/GSE147424_samples.csv' \
  -isave_intermediates='true' \
  -icelldex_reference='project-xxx:/scdownstream/singleR/celldex_references_example.csv' \
  -inextflow_soft_confs='project-xxx:/scdownstream/soft_configs/soft_config_test.config' \
  -inextflow_run_opts='-profile docker' \
  --destination 'project-xxx:/scdownstream/examples_results' \
  --priority high \
  --name "scdownstream_test_profile" \
  -y

Full test (GPU)

For the remaining runs, only the command line is provided. Refer to the Minimal test section for UI instructions.

The full test profile validates the pipeline with a complete tool selection and GPU support. Use soft_config_test_full.config in the soft_configs folder which includes GPU instance configuration for process_gpu labelled processes.

dx run project-xxx:/scdownstream/scdownstream_753add5 \
  -ioutdir='test_full_profile' \
  -iinput='project-xxx:/scdownstream/datasets/GSE147424_samples.csv' \
  -isave_intermediates='true' \
  -icelldex_reference='project-xxx:/scdownstream/singleR/celldex_references_example.csv' \
  -inextflow_soft_confs='project-xxx:/scdownstream/soft_configs/soft_config_test_full.config' \
  -inextflow_run_opts='-profile docker' \
  --destination 'project-xxx:/scdownstream/examples_results' \
  --priority high \
  --name "scdownstream_test_full_profile" \
  -y

Run with GSE171555

Several tools in the pipeline support GPU acceleration (scvitools, scimilarity, cellbender), so we used soft_config_GSE171555_gpu.config to provide GPU resources.

dx run project-xxx:/scdownstream/scdownstream_753add5 \
  -ioutdir='test_GSE171555' \
  -iinput='project-xxx:/scdownstream/datasets/GSE171555_samples.csv' \
  -isave_intermediates='true' \
  -icelldex_reference='project-xxx:/scdownstream/singleR/celldex_references_example.csv' \
  -inextflow_soft_confs='project-xxx:/scdownstream/soft_configs/soft_config_GSE171555_gpu.config' \
  -inextflow_run_opts='-profile docker' \
  --destination 'project-xxx:/scdownstream/examples_results' \
  --priority high \
  --name "scdownstream_GSE171555" \
  -y

Replace project-xxx with your DNAnexus project ID or Public Dataset project.

Run with GSE174609

• We used 12 samples from GSE174609 in nf-core/scrnaseq to process raw sequencing data. In the result, we have .h5ad files for each sample.

• In nf-core/scdownstream, gene symbols default to the index if not specified, which can cause errors in CellTypist if the index lacks gene names. In this case (from nf-core/scrnaseq), gene names are in the gene_symbol column, so it must be specified in the sample sheet.

• For differential expression across conditions (healthy, pre- and post-treatment), the condition must be included in both the .h5ad files (adata.obs) and the sample sheet (condition_col). Please refer to Considerations section for further information

• Command line to run this test. Replace project-xxx with your DNAnexus project ID or Public Dataset project.

dx run project-xxx:/scdownstream/scdownstream_753add5 \
  -ioutdir='test_GSE174609' \
  -iinput='project-xxx:/scdownstream/datasets/GSE174609_samples.csv' \
  -isave_intermediates='true' \
  -icelldex_reference='project-xxx:/scdownstream/singleR/celldex_references_example.csv' \
  -inextflow_soft_confs='project-xxx:/scdownstream/soft_configs/soft_config_GSE174609_gpu_qsize.config' \
  -inextflow_run_opts='-profile docker' \
  --destination 'project-xxx:/scdownstream/example_results' \
  --priority high \
  --instance-type mem2_ssd1_v2_x16 \
  --name "scdownstream_GSE174609_add_condition" \
  -y

Understanding the output

The output folder is organized as follows:

test_GSE174609/
├── celltypes/
├── cluster_dimred/
├── combine/
├── finalized/
├── multiqc/
├── per_group/
├── pipeline_info/
├── quality_control/
├── unify/
└── nextflow-job-scdownstream_GSE174609_add_condition.log

The finalized/ directory contains the final pipeline results, including merged.h5ad (the integrated .h5ad file with all results) and merged_metadata.csv (the corresponding metadata). Please refer to the nf-core/scdownstream official documentation for a full description of all output files.

An example notebook for exploring pipeline results is available on the platform: GSE174609_scdownstream_result_explore.ipynb in the notebook folder.

Considerations

Sample sheet issue

Gene symbol

If using inputs from nf-core/scrnaseq, verify which column in the .var table contains gene names and set it as symbol_col in the sample sheet. By default, nf-core/scdownstream uses the index as gene symbols; if the index lacks gene names, this can cause errors in CellTypist.

Condition information

For differential gene expression across conditions (healthy, pre-treatment, post-treatment), ensure the condition is included in both:

• files (adata.obs)

• sample sheet (condition_col)

Using scANVI

scANVI is a semi-supervised integration method that uses existing cell type annotations to guide integration. If your input data doesn't already have cell type labels, don't use scANVI. If you do have annotations, make sure your label_col contains at least two unique labels other than the unknown_label value. By default, if the label_col column doesn't exist in your input data, the pipeline creates one and sets all values to unknown, which will cause scANVI to fail.

CellTypist Model Download

In this pipeline, the cell type annotation step is performed using CellTypist within the CELLTYPES_CELLTYPIST process. The model specified (e.g., Adult_Human_Skin) is not embedded directly in the pipeline or container image. Instead, it is dynamically retrieved at runtime.

According to the execution logs, CellTypist first queries a remote model repository:

https://celltypist.cog.sanger.ac.uk/models/models.json

This endpoint provides a list of available pretrained models. Based on the specified model name, the pipeline then downloads the corresponding file (Adult_Human_Skin.pkl) and stores it locally in a temporary directory (e.g., ./tmp/celltypist/data/models).

Pulling docker image community.wave.seqera.io/library/celltypist_scanpy:44b604b24dd4cf33 : Attempt 1 out of 3...
📜 Retrieving model list from server https://celltypist.cog.sanger.ac.uk/models/models.json
📚 Total models in list: 61
📂 Storing models in ./tmp/celltypist/data/models
💾 Total models to download: 1
💾 Downloading model [1/1]: Adult_Human_Skin.pkl
🔬 Input data has 9349 cells and 9887 genes
🔗 Matching reference genes in the model
🧬 938 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!

This behavior indicates that CellTypist relies on an external model server and performs on-demand downloading of pretrained models. There is no parameter in the pipeline to put the server link or file path directly. There is only model parameter: –celltypist_model

Queue Size Configuration

The queueSize parameter controls how many tasks are executed in parallel by the Nextflow executor. On DNAnexus, this corresponds to the number of subjobs created simultaneously by the pipeline’s head job (default: 5). If you have more than 5 samples and want to allow more parallel jobs, include queueSize in the soft configuration file. For example: queueSize = 20 allows up to 20 subjobs to run in parallel

executor {
    queueSize = 20
}

Head job storage/memory issue

Due to the large number of child jobs, the default instance for the head job may run into a low-storage error, especially when you choose to save intermediate results. The error message is:

Cause of Failure
Please consult the job log; the job's reported error could not be parsed from the 
file job_error.json Warning: Low disk space during this job.

Solution: Use an instance with higher storage for the head job instead of the default instance. When running the pipeline with dx run, specify a larger instance using the --instance-type option.

GPU instance

Example configuration files for GPU instance: soft_config_GSE174609_gpu_qsize.config and soft_config_GSE171555_gpu.config

There are two types of driver: R470 and R535 available on DNAnexus and nf-core/scdownstream only works with R535. Please find and select instance with NVIDIA Driver R535 Compatibility

DNAnexus Instance AWS region	Cloud Instance
mem2_ssd1_gpu_x16	g4dn.4xlarge
mem2_ssd1_gpu_x32	g4dn.8xlarge
mem2_ssd1_gpu1_x32	g4dn.8xlarge
mem2_ssd1_gpu_x48	g4dn.12xlarge
mem2_ssd1_gpu4_x48	g4dn.12xlarge
mem2_ssd1_gpu_x64	g4dn.16xlarge
mem2_ssd1_gpu1_x64	g4dn.16xlarge
mem2_ssd2_gpu1_x4	g5.xlarge
mem2_ssd2_gpu1_x8	g5.2xlarge
mem2_ssd2_gpu1_x16	g5.4xlarge
mem2_ssd2_gpu1_x32	g5.8xlarge
mem2_ssd2_gpu1_x64	g5.16xlarge
mem2_ssd2_gpu4_x48	g5.12xlarge
mem2_ssd2_gpu4_x96	g5.24xlarge
mem2_ssd2_gpu8_x192	g5.48xlarge
mem2_ssd2_gpu1_v2_x4	g6.xlarge
mem2_ssd2_gpu1_v2_x8	g6.2xlarge
mem2_ssd2_gpu1_v2_x16	g6.4xlarge
mem2_ssd2_gpu1_v2_x32	g6.8xlarge
mem2_ssd2_gpu1_v2_x64	g6.16xlarge
mem2_ssd2_gpu4_v2_x48	g6.12xlarge
mem2_ssd2_gpu4_v2_x96	g6.24xlarge
mem2_ssd2_gpu8_v2_x192	g6.48xlarge
mem3_ssd1_gpu_x8_fedramp	g6.4xlarge
mem3_ssd1_gpu1_x16	gr6.4xlarge
mem3_ssd1_gpu1_x32	gr6.8xlarge
mem3_ssd2_gpu8_x192	p5.48xlarge