feat(core): improve performance of ExecutorService.handleChildWorkerT…

[loicmathieu]

…askResult

Searching for a retry in all parents is a costly operation, doing it only wgen we are retrying or failing avoid it most of the time.

Tested on the following flow

id: outputs
namespace: company.team

tasks:
  - id: forEach1
    type: io.kestra.plugin.core.flow.ForEach
    values: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
    tasks:
    - id: forEach2
      type: io.kestra.plugin.core.flow.ForEach
      values: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
      tasks:
      - id: forEach3
        type: io.kestra.plugin.core.flow.ForEach
        values: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
        tasks:
        - id: output
          type: io.kestra.plugin.core.output.OutputValues
          values:
            attr: "{{parents[1].taskrun.value}}-{{parent.taskrun.value}}-{{taskrun.value}}"
        - id: log
          type: io.kestra.plugin.core.log.Log
          message: "{{currentEachOutput(outputs.output).values.attr}}"

Execution go from 28mn to 24mn ( around 12% improvements)

Raw performance improvements number for processing an execution message (ExecutorService.process):

• CPU samples go from 10% to 6%
• Allocation samples go from 21% to 10%

[codecov]

❌ 111 Tests Failed:

Tests completed	Failed	Passed	Skipped
1963	111	1852	11

View the top 3 failed test(s) by shortest run time

io.kestra.runner.h2.H2RunnerRetryTest retryNewExecutionFlowDuration()

Stack Traces | 15s run time

java.util.concurrent.TimeoutException: Await failed to terminate within PT15S.
	at io.kestra.core.utils.Await.until(Await.java:42)
	at io.kestra.core.utils.Await.until(Await.java:31)
	at io.kestra.core.runners.RunnerUtils.awaitExecution(RunnerUtils.java:161)
	at io.kestra.core.runners.RunnerUtils.runOne(RunnerUtils.java:88)
	at io.kestra.core.runners.RunnerUtils.runOne(RunnerUtils.java:64)
	at io.kestra.core.runners.RunnerUtils.runOne(RunnerUtils.java:52)
	at io.kestra.plugin.core.flow.RetryCaseTest.retryNewExecutionFlowDuration(RetryCaseTest.java:143)
	at io.kestra.jdbc.runner.JdbcRunnerRetryTest.retryNewExecutionFlowDuration(JdbcRunnerRetryTest.java:72)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at io.micronaut.test.extensions.junit5.MicronautJunit5Extension$2.proceed(MicronautJunit5Extension.java:142)
	at io.micronaut.test.extensions.AbstractMicronautExtension.interceptEach(AbstractMicronautExtension.java:162)
	at io.micronaut.test.extensions.AbstractMicronautExtension.interceptTest(AbstractMicronautExtension.java:119)
	at io.micronaut.test.extensions.junit5.MicronautJunit5Extension.interceptTestMethod(MicronautJunit5Extension.java:129)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)

io.kestra.runner.h2.H2RunnerRetryTest retryNewExecutionTaskDuration()

Stack Traces | 15s run time

java.util.concurrent.TimeoutException: Await failed to terminate within PT15S.
	at io.kestra.core.utils.Await.until(Await.java:42)
	at io.kestra.core.utils.Await.until(Await.java:31)
	at io.kestra.core.runners.RunnerUtils.awaitExecution(RunnerUtils.java:161)
	at io.kestra.core.runners.RunnerUtils.runOne(RunnerUtils.java:88)
	at io.kestra.core.runners.RunnerUtils.runOne(RunnerUtils.java:64)
	at io.kestra.core.runners.RunnerUtils.runOne(RunnerUtils.java:52)
	at io.kestra.plugin.core.flow.RetryCaseTest.retryNewExecutionTaskDuration(RetryCaseTest.java:89)
	at io.kestra.jdbc.runner.JdbcRunnerRetryTest.retryNewExecutionTaskDuration(JdbcRunnerRetryTest.java:60)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at io.micronaut.test.extensions.junit5.MicronautJunit5Extension$2.proceed(MicronautJunit5Extension.java:142)
	at io.micronaut.test.extensions.AbstractMicronautExtension.interceptEach(AbstractMicronautExtension.java:162)
	at io.micronaut.test.extensions.AbstractMicronautExtension.interceptTest(AbstractMicronautExtension.java:119)
	at io.micronaut.test.extensions.junit5.MicronautJunit5Extension.interceptTestMethod(MicronautJunit5Extension.java:129)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)

io.kestra.runner.postgres.PostgresRunnerRetryTest retryNewExecutionFlowAttempts()

Stack Traces | 15s run time

java.util.concurrent.TimeoutException: Await failed to terminate within PT15S.
	at io.kestra.core.utils.Await.until(Await.java:42)
	at io.kestra.core.utils.Await.until(Await.java:31)
	at io.kestra.core.runners.RunnerUtils.awaitExecution(RunnerUtils.java:161)
	at io.kestra.core.runners.RunnerUtils.runOne(RunnerUtils.java:88)
	at io.kestra.core.runners.RunnerUtils.runOne(RunnerUtils.java:64)
	at io.kestra.core.runners.RunnerUtils.runOne(RunnerUtils.java:52)
	at io.kestra.plugin.core.flow.RetryCaseTest.retryNewExecutionFlowAttempts(RetryCaseTest.java:170)
	at io.kestra.jdbc.runner.JdbcRunnerRetryTest.retryNewExecutionFlowAttempts(JdbcRunnerRetryTest.java:78)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at io.micronaut.test.extensions.junit5.MicronautJunit5Extension$2.proceed(MicronautJunit5Extension.java:142)
	at io.micronaut.test.extensions.AbstractMicronautExtension.interceptEach(AbstractMicronautExtension.java:162)
	at io.micronaut.test.extensions.AbstractMicronautExtension.interceptTest(AbstractMicronautExtension.java:119)
	at io.micronaut.test.extensions.junit5.MicronautJunit5Extension.interceptTestMethod(MicronautJunit5Extension.java:129)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

[sonarqubecloud]

Quality Gate failed

Failed conditions
2 Security Hotspots
63.5% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud