Introduction

Building Applets

DNAnexus apps and applets are ways to package executable code. The biggest difference between apps and applets is their visibility. Apps such as you find in the Tool Library are globally available and maintained by DNAnexus and partners like Nvidia and Sentieon. Applets are private to an organization and exist as data objects in a project. They can be shared across projects and promoted to generally available apps. Native DNAnexus applets are built using dx build to create an executable for bash or Python code, which in turn may execute any program installed on the instance.

Later, we will discuss how to build a workflow, which is a combination of two or more apps/applets. We will build native workflows using the GUI and languages like WDL (Workflow Description Language) and Nextflow combine with Docker images.

Development Cycle

As shown in following figure, the development cycle is to write code locally, use dx build to create a native applet on the platform, and then dx run to run the applet. You can view the execution logs with dx watch, then make changes to your code to build and run again.

Installing Required Software

To install the Python modules required for this tutorial, run the following command:

python3 -m pip install dxpy miniwdl

You may be prompted to expand PATH with installation directory such as ~/.local/bin:

PATH=~/.local/bin:$PATH

Next, ensure you have a recent version of Java. For this tutorial, I'm using the following:

$ javac -version
javac 18

If you want to use Cromwell to execute WDL locally, you should download the Cromwell Jar file. This tutorial assumes you will place this file in your home directory using the following commands.

cd ~
wget https://github.com/broadinstitute/cromwell/releases/download/84/cromwell-84.jar

I suggest you use the link command (ln) to create a symlink to the filename cromwell.jar so that upgrading in the future will not break your commands:

ln -s cromwell-84.jar cromwell.jar

WOMtool (Workflow Object Model) is also quite useful, and I suggest you similarly download it and link it to womtool.jar:

cd ~
wget https://github.com/broadinstitute/cromwell/releases/download/84/womtool-84.jar
ln -s womtool-84.jar womtool.jar

You will use the DNAnexus dxCompiler to build WDL applications on the platform. Find a link to the latest Jar file under the releases of the Git repository. For example, the following commands will download dxCompiler-2.10.4.jar to your home directory and symlink it to dxCompiler.jar:

cd ~
wget https://github.com/dnanexus/dxCompiler/releases/download/2.10.5/dxCompiler-2.10.5.jar
ln -s dxCompiler-2.10.5.jar dxCompiler.jar

Some tools may attempt to use the shellcheck tool to validate any shell code in your WDL. To install on Ubuntu, run the following:

sudo apt install shellcheck

On macOS, you can use Homebrew to install the program:

brew install shellcheck

The dx CLI

If the dxpy module installed properly, you should able to run dx on the command line. For instance, run dx all to see a list of valid commands:

$ dx all
usage: dx [-h] [--version] command ...

DNAnexus Command-Line Client, API v1.0.0, client v0.320.0

dx is a command-line client for interacting with the DNAnexus platform.  You
can log in, navigate, upload, organize and share your data, launch analyses,
and more.  For a quick tour of what the tool can do, see

  https://documentation.dnanexus.com/getting-started/tutorials/cli-quickstart#quickstart-for-cli

For a breakdown of dx commands by category, run "dx help".

dx exits with exit code 3 if invalid input is provided or an invalid operation
is requested, and exit code 1 if an internal error is encountered.  The latter
usually indicate bugs in dx; please report them at

  https://github.com/dnanexus/dx-toolkit/issues

optional arguments:
  -h, --help  show this help message and exit
  --env-help  Display help message for overriding environment
              variables
  --version   show program's version number and exit

dx: error: argument command: invalid choice: all
(choose from login, logout, exit, whoami, env, setenv, clearenv, invite,
uninvite, ls, tree, pwd, select, cd, cp, mv, mkdir, rmdir, rm, describe,
upload, download, make_download_url, cat, head, build, build_asset, add, list,
remove, update, install, uninstall, run, watch, ssh_config, ssh, terminate,
rmproject, new, get_details, set_details, set_visibility, add_types,
remove_types, tag, untag, rename, set_properties, unset_properties, close,
wait, get, find, api, upgrade, generate_batch_inputs,
publish, archive, unarchive, help)

To get started, do the following:

• Run dx login to identify yourself to the DNAnexus platform. Enter your username and password. You can also set up a token to log in. Information on setting up tokens can be found in the Using Tokens section of our Documentation.

• You may also be prompted to select a project. If not, you should use dx select to select a project that will contain your work.

• If you do not see a project you wish to use for your work, run dx new project to create one from the command line, or click "New Project" in the web interface.

• Finally, run dx ssh_config to set up SSH keys for connecting to cloud instances.

Note that each subcommand will respond to the flags -h|--help to display the usage documentation. For instance, dx new can create several object types, which you can discover by reading the documentation:

$ dx new -h
usage: dx new [-h] class ...

Use this command with one of the available subcommands (classes) to create a
new project or data object from scratch. Not all data types are supported. See
'dx upload' for files and 'dx build' for applets.

positional arguments:
  class
    user      Create a new user account
    org       Create new non-billable org
    project   Create a new project
    record    Create a new record
    workflow  Create a new workflow

optional arguments:
  -h, --help  show this help message and exit

You should now be prepared to develop DNAnexus apps and workflows.

Resources

Full Documentation

Please email [email protected] to create a support ticket if there are technical issues.