# Example 1: hello To begin, you'll code a "Hello, World!" workflow that captures the output of a command into a file. WDL syntax may look familiar if you know any C-family language like Java or Perl. For example, keywords like `workflow` and `task` are used to define blocks of statements contained inside matched curly braces (`{}`), and variables are defined using a data type like `String` or `File`. In this example, you will: * Write a simple workflow in WDL * Learn two ways to capture the standard out (`STDOUT`) of a `command` block ## Getting Started To see this in action, make a *hello* directory for your work, and inside that create the file *workflow\.wdl* with the following contents: ``` version 1.0 workflow hello_world { input { String name } call write_greeting { input: greet_name = name } } task write_greeting { input { String greet_name } command { echo 'Hello, ${greet_name}!' } output { File outfile = stdout() } } ``` * The [`version`](https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#versioning) states that the following WDL follows the [WDL 1.0](https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md) specification. * The [`workflow`](https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#workflow-definition) keyword defines a workflow name. The contents of the workflow are enclosed in matched curly braces. * The [`input`](https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#workflow-inputs) block describes the parameters for the workflow. * WDL defines several [data types](https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#data-types—​serialization) you can use to describe an input value. This workflow requires a `String` parameter called `name`. * [`call`](https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#call-statement) will execute the task named `write_greeting`. This similar to executing a function in code. * The `input` keyword allows you to pass arguments to the task. The workflow's `name` input will be passed as `greet_name` to the task. * The [`task`](https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#task-definition) keyword defines a task called `write_greeting`. * The task also defines an [`input`](https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#task-inputs) block with a parameter `greet_name`. It would be fine to call this `name` because it would not conflict with workflow's `name`. * The [`command`](https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#command-section) keyword defines a block of shell commands to execute. The block may be denoted with matched curly braces (`{}`) or triple angle brackets (`<<<`/`>>>`). * The shell command `echo` prints a salutation to *standard out*, AKA `STDOUT`, which is the normal place for output from command-line program to appear. The variable `greet_name` is interpolated in the string by surrounding it with `~{}` or `${}` because the `command` block uses curly braces. When using triple angle brackets, only the first syntax is valid. * The [`output`](https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#outputs-section) block defines an `outfile` variable of the type `File`. The contents of the file is the captured [`stdout`](https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#file-stdout) from the `command` block. WDL is not whitespace dependent, so indentation is based on your preference. In the Setup section, you should have installed the [miniwdl](https://miniwdl.readthedocs.io/en/latest/) tool, which can be useful to check the syntax of your WDL. The following command shows the output when there are no problems: ``` $ miniwdl check workflow.wdl workflow.wdl workflow hello_world call write_greeting task write_greeting ``` Introduce an error in your WDL to see how the output changes. For instance, change the `version` to `2.0` and observe the error message: ``` $ miniwdl check workflow.wdl (workflow.wdl Ln 0 Col 0) unknown WDL version 2.0; choices: draft-2, 1.0, development, 1.1 ``` Or change the `call` to `write_greetings`: ``` $ miniwdl check workflow.wdl (workflow.wdl Ln 8 Col 5) No such task/workflow: write_greetings call write_greetings { ^^^^^^^^^^^^^^^^^^^^^^ ``` Cromwell will also find this error, but the message will be one of literally thousands of lines of output. Note that `miniwdl` uses a different parser than dxCompiler, and each has slightly different ideas of what constitutes valid syntax. For example, `miniwdl` requires commas in between `input` items but dxCompiler does not. In spite of their differences, I appreciate the concise reporting of errors that `miniwdl` provides. ## Executing WDL locally with Cromwell To execute this workflow locally using Cromwell, you must first create a JSON file to define the input name. Create a file called *inputs.json* with the following contents if you'd like to extend salutations to my friend Geoffrey: ``` { "hello_world.name": "Geoffrey" } ``` Next, run the following command to execute the workflow: ``` $ java -jar ~/cromwell-82.jar run --inputs inputs.json workflow.wdl ``` The output will be copious and should include an indication that the command was successful and the output landed in a file in the *cromwell-executions* directory that was created: ``` { "hello_world.write_greeting.outfile": "/Users/kyclark@dnanexus.com/work/srna/wdl_tutorial/hello/ cromwell-executions/hello_world/7f02fe78-4aff-4e01-95da-c9b6e021773d/ call-write_greeting/execution/stdout" } ``` You can use the `cat` (*concatenate*) command to see the contents of the file. Be sure to change the file path to the one created by your execution: ``` $ cat cromwell-executions/hello_world/7f02fe78-4aff-4e01-95da-c9b6e021773d/call-write_greeting/execution/stdout Hello, Geoffrey! ``` Here is another way to write the `command` block and capture `STDOUT` to a named file: ``` version 1.0 workflow hello_world { input { String name } call write_greeting { input: greet_name = name } } task write_greeting { input { String greet_name } command <<< echo 'Hello, ~{greet_name}!' > out.txt >>> output { File outfile = "out.txt" } } ``` * The `command` block here uses triple angle brackets to enclose the shell commands. * The variable must be [interpolated](https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#outputs-section) with `~{}` because of the triple angle brackets. The Unix redirect operator `>` is used to send the `STDOUT` from `echo` into the file *out.txt*. * The `outfile` is set to the file *out.txt*, which is created by the `command` block. If you execute this version, the output should show that the file *out.txt* was created instead of the file *stdout*: ``` { "outputs": { "hello_world.write_greeting.outfile": "/Users/kyclark@dnanexus.com/work/srna/wdl_tutorial/hello/ cromwell-executions/hello_world/1dd3abd8-be70-418b-9a31-b4ea9d5add99/ call-write_greeting/execution/out.txt" }, "id": "1dd3abd8-be70-418b-9a31-b4ea9d5add99" } ``` I can use `cat` again to verify that the same file was created: ``` $ cat cromwell-executions/hello_world/1dd3abd8-be70-418b-9a31-b4ea9d5add99/ call-write_greeting/execution/out.txt Hello, Geoffrey! ``` ## Creating a WDL applet with dxCompiler Now that you have verified that the workflow runs correctly on your local machine, it's time to compile this onto the DNAnexus platform. First, create a project in your organization and take note of the project ID. I'll demonstrate using the `dx` command-line interface to create a project called *Workflow Test*: ``` $ dx new project "Workflow Test" Created new project called "Workflow Test" (project-GFbKy7Q0ff1k3fGq48ZFZ45p) Switch to new project now? [y/N]: y ``` All the `dx` commands will print help documentation if you supply the `-h` or `--help` flags. For instance, run **`dx new project --help`**. You can also use the web interface, in which case you should use `dx select` to switch to the project. Next, use dxCompiler to compile the workflow into a *workflows* directory in the new project. In the following command, the dxCompiler prints the new workflow ID upon success: ``` $ java -jar ~/dxCompiler-2.10.2.jar compile workflow.wdl -folder /workflows \ > -project project-GFbKy7Q0ff1k3fGq48ZFZ45p workflow-GFbP9480ff1zVQPG48zXpfzb ``` ## Running a Workflow from the Web Interface Use the web interface to inspect the new workflow as shown in Figure 1. Click on the info button (an "i" in a circle to the right of the "Run" button) to verify the workflow ID is the same as you see on the command line. ![](/files/QTP3PlUK8nIf9fH5Zfro) Use the "Run" button in the web interface to launch the applet as shown in Figure 2. As shown in Figue 2, I indicate the applet's outputs should written to the *outputs* directory. ![](/files/lfQWW0ZqAPgYYfVcFUSm) Click on the "Analysis Inputs" view to specify a name for the greeting. In Figure 3, you see I have selected the name "Jonas." ![](/files/sOAuQ5QCgDhCvc299jVk) Click "Start Analysis" to start the workflow. The web interface will show the progress of running the applet as shown in Figure 4. ![](/files/NqWY7SuEg7VrXLNo9WyX) Figure 5 shows check marks next to each step that has been completed. Click the button to show inputs and outputs, then click on the link to the output file, which may be *stdout* or *out.txt* depending on the version of the workflow you compiled. ![](/files/ZTS3HhSolkRvngozLDar) Click on the output file name to view the contents of the file as shown in Figure 6. ![](/files/Wtf5X1FiH0qH587UxkAh) Click on the "Monitor" view to see how long the job lasted and cost as shown in Figure 7. ![](/files/cm9Bx1S8ClGUZNUpvewl) ## Running a Workflow from the Command Line You can also use the `dx` CLI to run the applet as shown in the following interactive session: ``` $ dx run workflow-GFbP9480ff1zVQPG48zXpfzb Entering interactive mode for input selection. Input: stage-common.name (stage-common.name) Class: string Enter string value ('?' for more options) stage-common.name: Ronald Select an optional parameter to set by its # (^D or to finish): [0] stage-common.overrides___ (stage-common.overrides___) [1] stage-common.overrides______dxfiles (stage-common.overrides______dxfiles) [2] stage-0.greet_name (stage-0.greet_name) [default={"$dnanexus_link": {"outputField": "name", "stage": "stage-common"}}] [3] stage-0.overrides___ (stage-0.overrides___) [4] stage-0.overrides______dxfiles (stage-0.overrides______dxfiles) [5] stage-outputs.overrides___ (stage-outputs.overrides___) [6] stage-outputs.overrides______dxfiles (stage-outputs.overrides______dxfiles) Optional param #: The following 1 stage(s) will reuse results from a previous analysis: Stage 2: outputs (job-GFbPJx80ff1gYQy5Fg3pK3GY) Using input JSON: { "stage-common.name": "Ronald" } Confirm running the executable with this input [Y/n]: y Calling workflow-GFbP9480ff1zVQPG48zXpfzb with output destination project-GFbKy7Q0ff1k3fGq48ZFZ45p:/ Analysis ID: analysis-GFbPjVj0ff1ZypqJ8vQj8kjZ ``` You can also specify the input JSON on the command line as a string or a file. In the following command, I provide the JSON as a string. Also note the use of the `-y` (*yes*) flag to have the workflow run without confirmation: ``` $ dx run workflow-GFbP9480ff1zVQPG48zXpfzb -j '{"stage-common.name": "Ronald"}' -y The following 3 stage(s) will reuse results from a previous analysis: Stage 0: common (job-GFbPjVj0ff1ZypqJ8vQj8kjf) Stage 1: write_greeting (job-GFbPjVj0ff1ZypqJ8vQj8kjg) Stage 2: outputs (job-GFbPJx80ff1gYQy5Fg3pK3GY) Using input JSON: { "stage-common.name": "Ronald" } Calling workflow-GFbP9480ff1zVQPG48zXpfzb with output destination project-GFbKy7Q0ff1k3fGq48ZFZ45p:/ Analysis ID: analysis-GFbPkFj0ff1k3fGq48ZFZ5Jy ``` You can also place the JSON into a file like so: ``` $ cat app_inputs.json {"stage-common.name": "Ronald"} ``` You can execute the workflow with this JSON file as follows: ``` $ dx run -f app_inputs.json workflow-GFbP9480ff1zVQPG48zXpfzb ``` You may also run the workflow with the `-h|--help` flag to see how to pass the arguments on the command line: ``` $ dx run workflow-GFbP9480ff1zVQPG48zXpfzb -h usage: dx run workflow-GFbP9480ff1zVQPG48zXpfzb [-iINPUT_NAME=VALUE ...] Workflow: hello_world Inputs: stage-common stage-common.name: -istage-common.name=(string) stage-common: Reserved for dxCompiler stage-common.overrides___: [-istage-common.overrides___=(hash)] stage-common.overrides______dxfiles: [-istage-common.overrides______dxfiles=(> stage-0 stage-0.greet_name: [-istage-0.greet_name=(string, default={"$dnanexus_link":> stage-0: Reserved for dxCompiler stage-0.overrides___: [-istage-0.overrides___=(hash)] stage-0.overrides______dxfiles: [-istage-0.overrides______dxfiles=(file) [-is> stage-outputs: Reserved for dxCompiler stage-outputs.overrides___: [-istage-outputs.overrides___=(hash)] stage-outputs.overrides______dxfiles: [-istage-outputs.overrides______dxfiles> Outputs: stage-common.name: stage-common.name (string) stage-0.outfile: stage-0.outfile (file) ``` For instance, you can also launch the app using the following command to greet "Keith": ``` $ dx run workflow-GFbP9480ff1zVQPG48zXpfzb -istage-common.name=Keith ``` However you choose to launch the workflow, the new run should be displayed in the "Monitor" view of the web interface. As shown in Figure 8, the new run finished in under 1 minute. ![](/files/oixePk1T7ZM5J8yVJXEQ) To find out more about the latest run, click on job's name in the run table. As shown in Figure 9, the platform will reuse files from the first run as it sees that nothing has changed. This is called "smart reuse," and you can disable this feature if you like. ![](/files/fDmpKcpVwfQ6HYK4RVVz) You can also use the CLI to view the results of the run with the `dx describe` command: ``` Result 1: ID analysis-GFbPjVj0ff1ZypqJ8vQj8kjZ Class analysis Job name hello_world Executable name hello_world Project context project-GFbKy7Q0ff1k3fGq48ZFZ45p Billed to org-sos Workspace container-GFbPjVj0ff1ZypqJ8vQj8kjb Workflow workflow-GFbP9480ff1zVQPG48zXpfzb Priority normal State done Root execution analysis-GFbPjVj0ff1ZypqJ8vQj8kjZ Parent job - Stage 0 common (stage-common) Executable applet-GFbP93j0ff1py9y87vzB2QQJ Execution job-GFbPjVj0ff1ZypqJ8vQj8kjf (done) Stage 1 write_greeting (stage-0) Executable applet-GFbP9380ff1XzVKkG9kyVg64 Execution job-GFbPjVj0ff1ZypqJ8vQj8kjg (done) Stage 2 outputs (stage-outputs) Executable applet-GFbP9400ff1pK6v113KJQF9g Execution [job-GFbPJx80ff1gYQy5Fg3pK3GY] (done) Cached from analysis-GFbPJx80ff1gYQy5Fg3pK3GP Input stage-common.name = "Ronald" [stage-0.greet_name = {"$dnanexus_link": {"analysis": "analysis-GFbPjVj0ff1ZypqJ8vQj8kjZ", "stage": "stage-common", "field": "name", "wasInternal": true}}] Output stage-common.name = "Ronald" stage-0.outfile = file-GFbPkBj0XFYgB7Vj4pF8XXBQ Output folder / Launched by kyclark Created Wed Aug 3 15:52:55 2022 Finished Wed Aug 3 15:54:51 2022 (Wall-clock time: 0:01:55) Last modified Wed Aug 3 15:54:54 2022 Depends on - Tags - Properties - Total Price $0.00 detachedFrom null rank 0 priceComputedAt 1659567291327 currency {"dxCode": 0, "code": "USD", "symbol": "$", "symbolPosition": "left", "decimalSymbol": ".", "groupingSymbol": ","} totalEgress {"regionLocalEgress": 0, "internetEgress": 0, "interRegionEgress": 0} egressComputedAt 1659567291327 costLimit null ``` Notice in the preceding output that the `Output` lists `file-GFbPkBj0XFYgB7Vj4pF8XXBQ`. You can `cat` the contents of this file with the CLI: ``` $ dx cat file-GFbPkBj0XFYgB7Vj4pF8XXBQ Hello, Ronald! ``` Alternately, you can download the file: ``` $ dx download file-GFbPkBj0XFYgB7Vj4pF8XXBQ [===========================================================>] Completed 15 of 15 bytes (100%) /Users/kyclark@dnanexus.com/work/srna/wdl_tutorial/stdout ``` The preceding command should create a new local file called *stdout* or *out.txt*, depending on the version of the workflow you compiled. Use the `cat` command again to verify the contents: ``` $ cat stdout Hello, Ronald! ``` ## Creating Command Shortcuts Using a Makefile You can create command-line shortcuts for all the steps of checking and buildingyour workflow by recording them as targets in a *Makefile* as follows: ``` WORKFLOW = workflow.wdl PROJECT_ID = project-GFPQvY007GyyXgXGP7x9zbGb DXCOMPILER = java -jar ~/dxCompiler-2.10.2.jar CROMWELL = java -jar ~/cromwell-82.jar check: miniwdl check $(WORKFLOW) local: $(CROMWELL) run --inputs inputs.json $(WORKFLOW) local2: $(CROMWELL) run workflow2.wdl app: $(DXCOMPILER) compile $(WORKFLOW) \ -archive \ -folder /workflows \ -project $(PROJECT_ID) clean: rm -rf cromwell-workflow-logs cromwell-executions ``` [GNU Make](https://www.gnu.org/software/make/) (or a similar Make program, which you may need to install) can turn the command `make local` into the listed Cromwell command to run one of the workflow versions. Makefiles are a handy way to document your work and automate your efforts. ## Review You should now be able to do the following: * Write a valid WDL workflow that accepts a string input and interpolates that string in a bash command. * Capture the standard output of a `command` block either using the `stdout()` WDL directive or by redirecting the output of a Unix command to a named file. * Define a `File` type `output` from a `task`. * Check the syntax of a WDL file using `miniwdl`. * Execute a WDL workflow on your local computer using Cromwell with the inputs defined in a JSON file. * Create a new project to contain a workflow. * Compile a WDL workflow into a DNAnexus applet using the the dxCompiler. * Run an applet using the web interface or the CLI. * Inspect the file output of an applet using the web interface or the CLI. * Download a file from the platform. * Use a *Makefile* to document and automate the steps for building and running a workflow. In the next section, you'll learn how to accept a file input and launch parallel processes to speed execution of large tasks. ## Review In this chapter, you learned some more WDL functions. ## Resources [Full Documentation](https://documentation.dnanexus.com/) To create a support ticket if there are technical issues: 1. Go to the Help header (same section where Projects and Tools are) inside the platform 2. Select "Contact Support" 3. Fill in the Subject and Message to submit a support ticket. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://academy.dnanexus.com/buildingworkflows/wdl/wdl_hello.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.