The Workflow Description Language (WDL) (pronounced as /hwɪdl/ or "whittle" with a 'd') is an open standard for describing data processing workflows using a human-readable/writeable syntax.
It introduces a domain-specific language that aims to provide simple but powerful facilities for (a) defining atomic units of computation (a task), (b) for connecting those atomic units together into a larger computation graph (a workflow), and (c) for effortlessly scaling the execution of these graphs in multiple environments.
Most fundamentally, WDL differentiates itself from other workflow languages by focusing on ease of implementation and accessibility to a wide range of users, including software engineers, domain experts (e.g., biologists), and operators of production computational systems. Furthermore, idiomatic workflow execution patterns, such as conditional execution, dynamic resource allocation, and scatter-gather operations, are simply expressed. Last, the language is driven and developed by a community of distributed volunteers as an open standard; there is no single execution engine, and anyone can build tooling to contribute to the ecosystem. Today, several engines and platforms exist and cover most execution environments (all major HPCs, all major clouds, local execution).
If you're new to WDL, welcome 👋. We highly recommend that you read the getting started documentation to learn how to write tasks and workflows.
The WDL community depends on your involvement to thrive. You are encouraged to ask questions, help other users, and make contributions where you can. Interactions occur primarily on GitHub and Slack. The WDL community also has an official website and blog that you can follow to stay up to date.
- If you have questions about using WDL, search in the discussions to see if the question has been asked already; if not, feel free to start a new discussion. You can also join the
#supportchannel on Slack to ask more transient questions. - If you are a user of an execution engine and encounter a bug or want to request a feature, you are encouraged to file an issue on the relevant execution engine's repository. Please do not file these types of questions/requests on this repository.
- If you are an implementor and need clarification on the specification or want to request a feature to be added to the WDL language, please file an issue on the WDL specification's GitHub issues page.
- If you'd like to contribute to the WDL specification, please read the contributing section before submitting a pull request.
The WDL specification contains all relevant information for users and developers, including those wanting to implement an execution engine. This GitHub project uses the branch for the current version of the specification as its primary branch, so you will always see the current version of the specification so long as you visit this project's root URL. Users are strongly encouraged to use the current version of the specification unless absolutely necessary.
- This branch is for version
1.2of the specification (link). All development of new non-breaking features should be done against this branch. - The next major version of the specification is version
2.0(link). All development of new breaking features should be done against that branch.
If desired, previous versions of the spec can be found on the GitHub branches page.
All products of the WDL project, including the specification, follow semantic versioning. That being said, the version of the WDL language and the version of the WDL specification are distinct:
-
The WDL language has a two-number version (e.g.,
1.2). An increase in the minor (second) version number (e.g.,1.1to1.2) indicates the addition of, or non-breaking changes to, the language or standard library functions. An increase in the major (first) version number (e.g.,1.0to2.0) indicates that breaking changes have been made. -
The WDL specification has a three-number version (e.g.,
1.2.0). The specification version tracks the language version, but there may also be patch releases (indicated by a change to the patch, or third, version number) that include fixes for typos, additional examples, or non-breaking clarifications of ambiguous language.
WDL only advances through community contributions. While participating in discussions and submitting issues are great ways to be involved, help is also needed to implement changes to the specification. For more information on how you can contribute, please read the Contributing guide.
The WDL specification is entirely community driven;
however, it is overseen by a Governance committee.
If you are interested in being involved in WDL governance, please join the Slack and post a message in the #general channel.
The WDL specification is made available to you under the permissive BSD 3-Clause License.
© 2024 The OpenWDL Developers