(Question) What does --latency-correction do?

[Vanav]

Test result without --latency-correction:

# ./oha -c 10 -t 5s -z 30s -q 10 --disable-keepalive https://example.com
Summary:
  Success rate: 1.0000
  Total:        30.0055 secs
  Slowest:      2.8308 secs
  Fastest:      0.7532 secs
  Average:      1.9747 secs
  Requests/sec: 5.5323

  Total data:   8.44 MiB
  Size/request: 52.06 KiB
  Size/sec:     288.00 KiB

Response time histogram:
  0.753 [1]  |
  0.961 [2]  |■
  1.169 [1]  |
  1.376 [3]  |■■
  1.584 [7]  |■■■■■
  1.792 [34] |■■■■■■■■■■■■■■■■■■■■■■■■
  2.000 [44] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  2.208 [34] |■■■■■■■■■■■■■■■■■■■■■■■■
  2.415 [23] |■■■■■■■■■■■■■■■■
  2.623 [8]  |■■■■■
  2.831 [9]  |■■■■■■

Latency distribution:
  10% in 1.6346 secs
  25% in 1.7788 secs
  50% in 1.9448 secs
  75% in 2.2062 secs
  90% in 2.4187 secs
  95% in 2.6259 secs
  99% in 2.7707 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.1100 secs, 0.0893 secs, 0.1347 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0009 secs

Status code distribution:
  [200] 166 responses

Results looks correct. I've checked server log, and response times are correctly reported.

Test result with --latency-correction as recommended:

# ./oha -c 10 -t 5s -z 30s -q 10 --latency-correction --disable-keepalive https://example.com
Summary:
  Success rate: 1.0000
  Total:        30.0014 secs
  Slowest:      31.7295 secs
  Fastest:      0.6359 secs
  Average:      16.3581 secs
  Requests/sec: 10.0329

  Total data:   15.30 MiB
  Size/request: 52.06 KiB
  Size/sec:     522.29 KiB

Response time histogram:
  0.636 [1]  |
  3.745 [28] |■■■■■■■■■■■■■■■■■■■■■■■■■■
  6.855 [32] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  9.964 [23] |■■■■■■■■■■■■■■■■■■■■■
  13.073 [33] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  16.183 [34] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  19.292 [28] |■■■■■■■■■■■■■■■■■■■■■■■■■■
  22.401 [32] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  25.511 [27] |■■■■■■■■■■■■■■■■■■■■■■■■■
  28.620 [33] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  31.729 [30] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■

Latency distribution:
  10% in 3.8071 secs
  25% in 8.8877 secs
  50% in 16.0519 secs
  75% in 24.4555 secs
  90% in 28.5860 secs
  95% in 29.9808 secs
  99% in 31.3885 secs

Details (average, fastest, slowest):
  DNS+dialup:   14.4690 secs, 0.0922 secs, 29.5267 secs
  DNS-lookup:   14.3596 secs, 0.0019 secs, 29.4168 secs

Status code distribution:
  [200] 301 responses

Test was twice longer than requested (60 s instead of 30 s), "Requests/sec" is incorrect: test lasted 60 s instead of 30 s, made 301 requests (correct), so it is 5 requests per second (confirmed by server logs). And response times are incorrect compared to server logs (response times were are the same as in previous test).

[hatoo]

Thanks for the issue!

Test was twice longer than requested (60 s instead of 30 s), "Requests/sec" is incorrect: test lasted 60 s instead of 30 s, made 301 requests (correct), so it is 5 requests per second (confirmed by server logs).

I see the test lasted for 30 s because the log shows Total: 30.0014 secs. How did you measure the time to take the test?

And response times are incorrect compared to server logs (response times were are the same as in previous test).

--latency-correction simulates a user sending requests -q qps even when a user can't receive responses at that rate.
So if the results show a longer response time than the server log, it's OK because a log is started by when the user wants to connect (not actually connected).

I've found that in --latency-correction the DNS+dialup and DNS-lookup become incorrect. I'll fix it.

[Vanav]

I see the test lasted for 30 s because the log shows Total: 30.0014 secs. How did you measure the time to take the test?

1. First line of tui shows strange: 60 s / 30 s — counting after it should stop
2. Logs on server: first line on 06:29:18, last line on 06:30:19

From server logs I got the same response times in both tests. So I still don't get why reported stats are different.

[hatoo]

@Vanav
I've fixed the duration of a test.
Could you check it with the master branch?

[thukabjj]

@hatoo once you've fixed this issue a log time ago and you did not receive the feedback from @Vanav I do believe that you should closed it, what do you think?
I am just recommend it, because I like to see the most updates issues to see if is there any opportunity that I can contribute.
Although, it's just a suggestion

[AlJohri]

On this topic I had two requests and one question:

1. It would be helpful to add a few details to the README addressing the original question of What does --latency-correction do? along with a brief explanation of Coordinated Omission and how this project attempts to solve it.

2. Perhaps behind a flag, can we output both the corrected and uncorrected latencies? You can see that wrk2 from Gil Tene does this here.

3. When I first saw this flag, I assumed that we would be handling the coordinated omission problem by sending requests at a fixed rate and awaiting the responses in a separate thread(s) so that the load generator continues sending requests to the server at the requested rate independent of the response times (i.e. an open workload generator). But it seems like we have a closed workload generator with a fixed number of workers/connections. My understanding is that we 1) emit tasks to a queue at a certain QPS and 2) have a defined set of consumers pulling tasks off of the queue and running them in a blocking manner. In this model, we handle the coordinated omission problem by keeping track of when the request should have been submitted and use this as the start time when calculating latency.

   I was wondering if you considered an alternative design of an open workload generator where you submit the requests asynchronously, independent of the response times?

Please correct me if my understanding of any of the above is incorrect. Thanks for your time!

[hatoo]

Thank you for your comment!
Honestly, I don't think I have a solid understanding of this topic.
I hadn't found good material about it...

1. It would be helpful to add a few details to the README addressing the original question of What does --latency-correction do? along with a brief explanation of Coordinated Omission and how this project attempts to solve it.

Yes, It would be good. I didn't just because I didn't feel like it.

2. Perhaps behind a flag, can we output both the corrected and uncorrected latencies? You can see that wrk2 from Gil Tene does this here.

It's a good idea. I want it behind a flag because it'll very long output.

When I first saw this flag, I assumed that we would be handling the coordinated omission problem by sending requests at a fixed rate and awaiting the responses in a separate thread(s) so that the load generator continues sending requests to the server at the requested rate independent of the response times (i.e. an open workload generator). But it seems like we have a closed workload generator with a fixed number of workers/connections. My understanding is that we 1) emit tasks to a queue at a certain QPS and 2) have a defined set of consumers pulling tasks off of the queue and running them in a blocking manner. In this model, we handle the coordinated omission problem by keeping track of when the request should have been submitted and use this as the start time when calculating latency.

I was wondering if you considered an alternative design of an open workload generator where you submit the requests asynchronously, independent of the response times?

I think our current model behind --latency-correction makes sense because it actually corrects latency by modifying the started timestamp.

I didn't know about open workload generator at the time when I implemented --latency-correction.
But it is also a reasonable choice and it's worth to implement.