Technical Considerations

This section applies to all nf-core input CSV builder applets.

Validation does not block export

The applet validates all inputs and logs warnings or errors, but does not prevent users from exporting the output samplesheet even if it contains invalid values. This means a job can complete successfully with warnings while the output samplesheet is still invalid for pipeline use. Therefore, please always review the log file for any warnings or errors before using the output samplesheet to run the pipeline.

Input Source Priority

When both a prefilled samplesheet and file inputs (FASTA files for proteinfold, FASTQ files or folder for scrnaseq) are provided simultaneously, the applet prioritizes the prefilled samplesheet and ignores the file inputs. To avoid unexpected behavior, users should choose only one input source.

For example, if a prefilled samplesheet with 5 rows is uploaded alongside 2 selected FASTA files, the output samplesheet will contain 5 rows from the prefilled samplesheet only. The 2 FASTA files will be ignored and the applet will log a warning.

Consistent input length

When providing manual input, all columns must have the same number of values. The only exception is a single constant value, which will be broadcast across all rows. For example, if 3 sample names are provided, all other columns must also have 3 values or a single constant. If not, the applet will log the following error

Manual input value of column '{col}' must be a constant or an array with the length of {expected_length} instead of {len(values)}.

CSV field count mismatch

When preparing a prefilled samplesheet, ensure that each row has the same number of fields as the header. A common mistake is having extra or missing commas in a row, which causes the applet to log the following error:

Prefilled samplesheet row (0): Header (3) and Data (4) lengths do not match.

Missing required columns: sample

Missing required columns: fastq_1

Missing required columns: fastq_2

Note that the Missing required columns errors are a consequence of the mismatch and do not indicate that the columns are actually missing. It is recommended to open the CSV file to verify the column alignment.

Empty cell in required columns

Leaving a cell empty in a required column (e.g. fastq_1) is treated differently from omitting the column entirely, but both result in similar error messages. When an empty cell is detected, the applet logs the following errors:

(0,fastq_1): File not found: nan

(0,fastq_1): FastQ file for reads 1 must be provided, cannot contain spaces and must have extension '.fq.gz' or '.fastq.gz'

Missing required columns: fastq_1

The output samplesheet will be generated but the affected column will be dropped. Ensure all required columns are fully populated before running the pipeline.

Sample name auto-detection

When no sample name is provided, the applet automatically extracts the sample name from the fastq_1 filename by taking the substring from the beginning of the filename to the first underscore. For example:

File name	Extracted sample name
AEG588A1_S1_L002_R1_001.fastq.gz	AEG588A1
SRR14575500_1.fastq.gz	SRR14575500
pbmc8k_S1_L007_R1_001.fastq.gz	pbmc8k

The only requirement is that the filename must contain at least one underscore _. If the filename does not contain an underscore, the applet will not be able to extract the sample name and will log a warning. In this case, it is recommended to provide the sample name manually via the sample field or use a prefilled samplesheet.

DNAnexus path resolution

The applet automatically resolves all file references into valid dx:// URIs in the output samplesheet. If a file path is provided in project-xxx:/path/to/file format (without dx:// prefix), the applet will automatically prepend dx://. For example:

Input format	Output in samplesheet
file-xxxxxxxxx	dx://project-xxx:/path/to/file.fastq.gz
project-xxx:/path/to/file.fastq.gz	dx://project-xxx:/path/to/file.fastq.gz
dx://project-xxx:/path/to/file.fastq.gz	dx://project-xxx:/path/to/file.fastq.gz
filename.fastq.gz	dx://project-xxx:/path/to/filename.fastq.gz

Using files from another project

The applet resolves file references within the project it is launched from. If a filename or relative path is provided, the applet will search for the file within the current project only. Files from other projects will not be found and the applet will log a File not found error.

To use files from another project, you must explicitly specify the project ID in one of the following formats:

project-xxxxxxxxx:file-xxxxxxxxx

project-xxxxxxxxx:/path/to/file.fastq.gz

dx://project-xxxxxxxxx:/path/to/file.fastq.gz

For example, if you are launching the applet from project-A but your FASTQ files are stored in project-B, providing only the filename will result in a File not found error even if the file exists in project-B. You must provide the full path with project-B ID explicitly.

Note: Make sure you have read access to the other project.

You can also select files in another project from UI.

• Click on All projects to view all projects you have access to.

• Then search a project by using project name or project ID.

Use files from public project

If you cannot find public projects in All projects, you need to prepare a prefilled sample sheet file with full path including project ID of public project.

For example, we want to use FASTA files in Public Project AWS East (project-J3JyY6j030gzQypGpk273241).

sequence,fasta,comment
T1024,project-J3JyY6j030gzQypGpk273241:/ProteinFold/samplesheet_input/Fasta/T1024.fasta,need to have absolute path for files in public project
T1026,project-J3JyY6j030gzQypGpk273241:/ProteinFold/samplesheet_input/Fasta/T1026.fasta,need to have absolute path for files in public project

Output file naming

If output_samplesheet_name is not provided, the applet defaults to sample_sheet.csv. If a name is provided without the .csv extension, the applet automatically appends it. For example:

Input	Output file name
my_samplesheet	my_samplesheet.csv
my_samplesheet.csv	my_samplesheet.csv
empty	sample_sheet.csv

Run applet with dx run

Here is an example of how to use dx run:

dx run $APPLET_ID --priority high -y \

-iversion="1.1.1" \

-isequence="T1024_2" \

-ifasta="file-J81Fz6j09QzKyF7xyv04vkKf" \

-isequence="T1026_2" \

-ifasta="file-J81Fz6j09Qz55XVJ9fgQyx0f" \

-ioutput_samplesheet_name="samplesheet_proteinfold.csv" \

--destination "project-J5Xqzy009Qz80FqQpy7QGKfV:/users/piyapong/test_applet/output/" \

--name "Test proteinfold v 1.1.1 with manual input sequence and fasta"

samplesheet_proteinfold.csv:

| | sequence | fasta |

|---:|:-----------|:-----------------------------------------------------------------------------------------------|

| 0 | T1024_2 | dx://project-J5Xqzy009Qz80FqQpy7QGKfV:/users/piyapong/test_applet/fasta/dataset2/T1024_2.fasta |

| 1 | T1026_2 | dx://project-J5Xqzy009Qz80FqQpy7QGKfV:/users/piyapong/test_applet/fasta/dataset2/T1026_2.fasta |

Limitations

The applet is designed as a helper tool for straightforward use cases. For the following scenarios, it is recommended to use the more capable JupyterLab notebook interface instead:

• Large studies with many samples.

• Complex multi-lane experiments.

• Cross-project file references requiring extensive path manipulation.

• Cases requiring custom metadata inference from filenames.

Schema drift

Here, the CSV Builder for nf-core/scrnaseq is built for version 4.1.0 and the CSV Builder for nf-core/proteinfold is built for version 1.1.1 and 2.0.0. If the pipeline is updated to a newer version with changes to required columns, validation rules, or accepted values, the applet may produce unexpected validation errors or incorrect output because it is still validating against the older schema.

Troubleshooting

Error message	Explanation	How to fix
Manual input value of column '{col}' must be a constant or an array with the length of {expected_length} instead of {len(values)}	The number of values provided for a column does not match the number of values in other columns.	Ensure all columns have the same number of values, or provide a single constant value to broadcast across all rows. See [Consistent Input Length] for more details.
Prefilled samplesheet row ({index}): Header ({len(header)}) and Data ({len(row)}) lengths do not match	The number of fields in a CSV row does not match the number of columns in the header, usually caused by extra or missing commas.	Open the CSV file to verify column alignment. See [CSV Field Count Mismatch] for more details
Missing required columns: {col}	A required column is completely missing from the prefilled samplesheet, or a required column contains an empty cell.	Add the missing column with valid values. If the column exists but has empty cells, ensure all cells are populated. See [Empty Cell in Required Column] for more details
({row_index},{col}): File not found: nan	A required cell is empty in the prefilled samplesheet.	Ensure all required columns are fully populated before uploading.
({row_index},{col}): FastQ file for reads 1 must be provided, cannot contain spaces and must have extension '.fq.gz' or '.fastq.gz'	A required FASTQ file is missing or has an invalid extension.	Provide a valid FASTQ file with extension .fq.gz or .fastq.gz.
({row_index},{col}): File name must contain 'R2'	The file assigned to fastq_2 does not contain R2 in its filename.	Check that the correct R2 file is assigned to fastq_2.
({row_index},{col}): Expect fastq_2 file name to be: {filename}	The applet detected a mismatch between the R1 and R2 filenames.	Check that the R1 and R2 files are correctly paired.
({row_index},{col}): Found {str(len(items))} files: {items}	Multiple files with the same name exist in the project, applet cannot determine which file to use.	Use a relative or absolute path, or file ID to uniquely identify the file. See [Ambiguous File Resolution] for more details.
({row_index},{col}): The entity {file_id} could not be found, code 404	The file ID provided does not exist in the project.	Check that the file ID is correct and the file exists in the project.
({row_index},{col}): Fasta file must be provided, cannot contain spaces and must have extension '.fa' or '.fasta'	A required FASTA file is missing or has an invalid extension.	Provide a valid FASTA file with extension .fa or .fasta.
Required column fasta cannot contain null	One or more cells in the fasta column are empty.	Ensure all rows in the fasta column have valid file references.
Require at least one of these columns: ['sequence', 'id']	Required columns are not provided.	Provide required columns for Proteinfold v1.1.1 (sequence) or v2.0.0 (id).
Found more than one additional required columns. Choose one of these columns as an input: Index(['sequence', 'id'], dtype='str')	Provide both id and sequence column while Proteinfold v1.1.1 or 2.0.0 requires only one.	Provide required columns for Proteinfold v1.1.1 (sequence) or v2.0.0 (id).