Example 1: hello
To begin, you'll code a "Hello, World!" workflow that captures the output of a command into a file. WDL syntax may look familiar if you know any C-family language like Java or Perl. For example, keywords like workflow and task are used to define blocks of statements contained inside matched curly braces ({}), and variables are defined using a data type like String or File.
In this example, you will:
Write a simple workflow in WDL
Learn two ways to capture the standard out (
STDOUT) of acommandblock
Getting Started
To see this in action, make a hello directory for your work, and inside that create the file workflow.wdl with the following contents:
The
inputkeyword allows you to pass arguments to the task. The workflow'snameinput will be passed asgreet_nameto the task.The shell command
echoprints a salutation to standard out, AKASTDOUT, which is the normal place for output from command-line program to appear. The variablegreet_nameis interpolated in the string by surrounding it with~{}or${}because thecommandblock uses curly braces. When using triple angle brackets, only the first syntax is valid.
WDL is not whitespace dependent, so indentation is based on your preference.
Introduce an error in your WDL to see how the output changes. For instance, change the version to 2.0 and observe the error message:
Or change the call to write_greetings:
Cromwell will also find this error, but the message will be one of literally thousands of lines of output.
Note that miniwdl uses a different parser than dxCompiler, and each has slightly different ideas of what constitutes valid syntax. For example, miniwdl requires commas in between input items but dxCompiler does not. In spite of their differences, I appreciate the concise reporting of errors that miniwdl provides.
Executing WDL locally with Cromwell
To execute this workflow locally using Cromwell, you must first create a JSON file to define the input name. Create a file called inputs.json with the following contents if you'd like to extend salutations to my friend Geoffrey:
Next, run the following command to execute the workflow:
The output will be copious and should include an indication that the command was successful and the output landed in a file in the cromwell-executions directory that was created:
You can use the cat (concatenate) command to see the contents of the file. Be sure to change the file path to the one created by your execution:
Here is another way to write the command block and capture STDOUT to a named file:
The
commandblock here uses triple angle brackets to enclose the shell commands.The
outfileis set to the file out.txt, which is created by thecommandblock.
If you execute this version, the output should show that the file out.txt was created instead of the file stdout:
I can use cat again to verify that the same file was created:
Creating a WDL applet with dxCompiler
Now that you have verified that the workflow runs correctly on your local machine, it's time to compile this onto the DNAnexus platform. First, create a project in your organization and take note of the project ID. I'll demonstrate using the dx command-line interface to create a project called Workflow Test:
All the dx commands will print help documentation if you supply the -h or --help flags. For instance, run dx new project --help.
You can also use the web interface, in which case you should use dx select to switch to the project. Next, use dxCompiler to compile the workflow into a workflows directory in the new project. In the following command, the dxCompiler prints the new workflow ID upon success:
Running a Workflow from the Web Interface
Use the web interface to inspect the new workflow as shown in Figure 1. Click on the info button (an "i" in a circle to the right of the "Run" button) to verify the workflow ID is the same as you see on the command line.
Use the "Run" button in the web interface to launch the applet as shown in Figure 2. As shown in Figue 2, I indicate the applet's outputs should written to the outputs directory.
Click on the "Analysis Inputs" view to specify a name for the greeting. In Figure 3, you see I have selected the name "Jonas."
Click "Start Analysis" to start the workflow. The web interface will show the progress of running the applet as shown in Figure 4.
Figure 5 shows check marks next to each step that has been completed. Click the button to show inputs and outputs, then click on the link to the output file, which may be stdout or out.txt depending on the version of the workflow you compiled.
Click on the output file name to view the contents of the file as shown in Figure 6.
Click on the "Monitor" view to see how long the job lasted and cost as shown in Figure 7.
Running a Workflow from the Command Line
You can also use the dx CLI to run the applet as shown in the following interactive session:
You can also specify the input JSON on the command line as a string or a file. In the following command, I provide the JSON as a string. Also note the use of the -y (yes) flag to have the workflow run without confirmation:
You can also place the JSON into a file like so:
You can execute the workflow with this JSON file as follows:
You may also run the workflow with the -h|--help flag to see how to pass the arguments on the command line:
For instance, you can also launch the app using the following command to greet "Keith":
However you choose to launch the workflow, the new run should be displayed in the "Monitor" view of the web interface. As shown in Figure 8, the new run finished in under 1 minute.
To find out more about the latest run, click on job's name in the run table. As shown in Figure 9, the platform will reuse files from the first run as it sees that nothing has changed. This is called "smart reuse," and you can disable this feature if you like.
You can also use the CLI to view the results of the run with the dx describe command:
Notice in the preceding output that the Output lists file-GFbPkBj0XFYgB7Vj4pF8XXBQ. You can cat the contents of this file with the CLI:
Alternately, you can download the file:
The preceding command should create a new local file called stdout or out.txt, depending on the version of the workflow you compiled. Use the cat command again to verify the contents:
Creating Command Shortcuts Using a Makefile
You can create command-line shortcuts for all the steps of checking and buildingyour workflow by recording them as targets in a Makefile as follows:
Review
You should now be able to do the following:
Write a valid WDL workflow that accepts a string input and interpolates that string in a bash command.
Capture the standard output of a
commandblock either using thestdout()WDL directive or by redirecting the output of a Unix command to a named file.Define a
Filetypeoutputfrom atask.Check the syntax of a WDL file using
miniwdl.Execute a WDL workflow on your local computer using Cromwell with the inputs defined in a JSON file.
Create a new project to contain a workflow.
Compile a WDL workflow into a DNAnexus applet using the the dxCompiler.
Run an applet using the web interface or the CLI.
Inspect the file output of an applet using the web interface or the
CLI.
Download a file from the platform.
Use a Makefile to document and automate the steps for building and running a workflow.
In the next section, you'll learn how to accept a file input and launch parallel processes to speed execution of large tasks.
Review
In this chapter, you learned some more WDL functions.
Resources
To create a support ticket if there are technical issues:
Go to the Help header (same section where Projects and Tools are) inside the platform
Select "Contact Support"
Fill in the Subject and Message to submit a support ticket.
Last updated
Was this helpful?
