Usability: Give users more control over process submissions

[mbercx]

Motivation

There are several use cases where a user will want to have more control over the processes submitted to AiiDA:

1. When running a large number of workflows, one might not want to submit all of them in one go but rather maintain a certain number of active workflows at any given time.
2. Since we typically only want to run a specific workflow once for each system, it's useful to be able to track which systems have already been run.

Desired Outcome

As discussed in the Progress section, there already is a tool available for controlling submissions in AiiDA. One desired outcome of this road map item is to better document this tool and make it findable for users.

(Optional) More generally, we might want to implement a feature for limiting the number of active jobs on a computer scheduler in aiida-core.

Impact

Any user that wants to run more workflows than their AiiDA setup or supercomputing facility can handle at one time will benefit from use case [1]. Maintaining a certain number of workflows automatically saves time submitting jobs manually and is efficient since the optimal number of workflows can remain active.

Most users will benefit from use case [2]. If you want to run a workflow once for a set of systems, keeping track of which systems have been run manually is typically a hassle.

Complexity

Implementing a basic tool for controlling submissions is quite straightforward and has already been done. The main work will be to fully document this package and point users to it from the AiiDA docs, which shouldn't be more than a couple days of dedicated work.

If we want to expand features in aiida-core to limit the number of active jobs on a scheduler, this would require changes to the AiiDA engine and hence be potentially much more complex.

Progress

There is already a tool available for controlling workflow submissions in AiiDA:

https://github.com/aiidateam/aiida-submission-controller

This approach relies on organising the workflows in groups and using uniquely defined extras to keep track of which systems were already run.

[chrisjsewell]

Thanks @mbercx,

Maybe change the title to: "Usability: Allow users more control over process submissions"

[sphuber]

Think the title is still quite a bit vage. Maybe it would help to have the word "rate" in there, as that is really the problem. We are trying to provide a built in mechanism to rate-limit the submission of processes in general and maybe to certain computers.