Retention module improvements

[mlditga]

Hello,

After trying the retention module we find the functionality very welcomed but we are wondering if some improvements could be considered in the future. It is especially about the fact that the instance selection/sorting/paging and filtering are separate.

What we would like to do:

• Set multiple sweepIntervals based on filter criteria (eg 15 days for finished workflows and 30 for faulted workflows)
• Stop the cleaning process as soon as no instance is found to delete (eg we had an issue with 48K instances in our database after running load tests. We set a batchSize of 100 and a timeToLive of 10 days but then we realized that only a very small number of instances where matching the filters. The query and filters where applied many times without deleting anything after the first iterations).
  [edit] this may be related to that issue : Retention module: paging may cause same instances to be selected at each iteration? #3062

Thank you for all your work!

[sfmskywalker]

Hi @mlditga

I like the suggestion! Please feel free to create an issue for the first bullet point.
The second item you mentioned should be solved now, but please let me know if otherwise.

Thanks!