Update benchmark warmup time

[ericywl]

Motivation/summary

Fixes #14948

Checklist

• Trim some words from benchmark workflow inputs to be more succinct
• Add benchmark warmup time to workflow inputs
• Increase benchmark warmup time for nightly non-standalone to 10m
• Reduce benchmark warmup time for scheduled standalone to 3m

How to test these changes

Run throughput benchmark

Appendix

Old workflow inputs:

New workflow inputs:

Test benchmark run on upstream

[mergify]

This pull request does not have a backport label. Could you fix it @ericywl? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-7.17 is the label to automatically backport to the 7.17 branch.
• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit.
• backport-8.x is the label to automatically backport to the 8.x branch.

[mergify]

backport-8.x has been added to help with the transition to the new branch 8.x.
If you don't need it please use backport-skip label.

[ericywl]

@rubvs From what I can gather, the benchmark warmup time is configured by BENCHMARK_WARMUP_TIME. Let me know if I missed anything. I will try to run the benchmarks and verify.

[rubvs]

@ericywl Correct. This should be the only change to make. You can move this out-of-draft and let's get feedback from the rest of the team.

With this issue there isn't much "testing" you can do. I've already concluded this through running many benchmarks. However, this is a great first-issue:

• It'll force you to read and hopefully improve the docs, see workflow.md in Throughput.
• Trigger a benchmark in GitHub Actions that run on ESS.
• Trigger a benchmark from your local machine that runs in ESS.
• Spot the env var change in the Terraform provisioning logs for confirmation.

[carsonip]

This is good for ESS, but probably not that helpful for standalone apm server (moxy) setup. Can we keep the 5m default for standalone apm setup?

[1pkg]

This is good for ESS, but probably not that helpful for standalone apm server (moxy) setup. Can we keep the 5m default for standalone apm setup?

We probably even should reduce it for standalone to not waste resources.

[ericywl]

This is good for ESS, but probably not that helpful for standalone apm server (moxy) setup. Can we keep the 5m default for standalone apm setup?

I'm not sure if there's a way to separate between the two. If not I will simply close the PR.

[carsonip]

I'm not sure if there's a way to separate between the two. If not I will simply close the PR.

The new default can be set in terraform files e.g. testing/benchmark/main.tf, just not in Makefile.

[carsonip]

Can you also test the PR by pushing to a branch to upstream apm-server repo and triggering workflow runs manually in the GH Actions UI?

[carsonip]

Other than the nits above, lgtm. Great work!