Feature: Improve graceful termination of the builder

[JKBGIT1]

Motivation

If the builder receives a SIGTERM while there is a running workflow, it first has to wait until all stages of the workflow are finished and only after that it can terminate. This graceful shutdown can be performed faster.

Description

In the new approach, the builder will wait until the current ongoing stage of the workflow finishes. After that, it terminates. In case that wasn't the last stage of the workflow, the builder re-runs the whole workflow.

An example:

1. builder starts a workflow
2. terraformer starts to provision the desired infrastructure
3. builder receives SIGTERM
4. terraformer hasn't finished the provisioning yet
5. builder waits until terraformer finishes the provisioning
6. terraformer finished provisioning
7. builder terminates
8. builder re-runs the whole workflow (including the terraformer part)

The re-run will move quickly through the terraformer stage because the infrastructure is already provisioned.

Exit criteria

• builder works with a new approach to SIGTERM handling.

Anything else to add

@bernardhalas as agreed, please complement me.