Emit more histogram buckets for cortex-querier_storegateway_refetches… #6570
Conversation
SungJin1212
force-pushed
the
Emit-more-buckets-for-cortex-querier_storegateway_refetches_per_query
branch
4 times, most recently
from
b50d72c to
e50e81d
Compare
| @@ -123,7 +123,7 @@ func newBlocksStoreQueryableMetrics(reg prometheus.Registerer) *blocksStoreQuery | |||
| Namespace: "cortex", | |||
| Name: "querier_storegateway_refetches_per_query", | |||
| Help: "Number of re-fetches attempted while querying store-gateway instances due to missing blocks.", | |||
| Buckets: []float64{0, 1, 2}, | |||
| Buckets: []float64{0, 1, 2, 4, 8, 16}, | |||
There was a problem hiding this comment.
We have a customized value of 5 retries. I don't know if it makes sense to have 16 retries. Shall we keep upper limit to 8?
There was a problem hiding this comment.
Can't we retry as many times as the user specifies? (https://github.com/cortexproject/cortex/blob/master/pkg/querier/blocks_store_queryable.go#L542)
There was a problem hiding this comment.
That's right. But I don't think we should recommend users to retry up to 16 times. I mean in general 3 to 7 retries make the most of sense. But maybe that's our usecase.
Do you have a usecase of using that many retries?
There was a problem hiding this comment.
Oh, you're right. let's fix it.
SungJin1212
force-pushed
the
Emit-more-buckets-for-cortex-querier_storegateway_refetches_per_query
branch
from
e50e81d to
b3d3e16
Compare
SungJin1212
force-pushed
the
Emit-more-buckets-for-cortex-querier_storegateway_refetches_per_query
branch
2 times, most recently
from
4880120 to
e85f1f9
Compare
CHANGELOG.md
Outdated
| @@ -3,6 +3,7 @@ | |||
| ## master / unreleased | |||
|
|
|||
| * [FEATURE] Querier/Ruler: Add `query_partial_data` and `rules_partial_data` limits to allow queries/rules to be evaluated with data from a single zone, if other zones are not available. #6526 | |||
| * [ENHANCEMENT] StoreGateway: Emit more histogram buckets on the `cortex-querier_storegateway_refetches_per_query` metric. #6570 | |||
There was a problem hiding this comment.
The name should be cortex_querier_storegateway_refetches_per_query
There was a problem hiding this comment.
Thanks for the catching.
…_per_query Signed-off-by: SungJin1212 <[email protected]>
SungJin1212
force-pushed
the
Emit-more-buckets-for-cortex-querier_storegateway_refetches_per_query
branch
from
e85f1f9 to
ed3a42a
Compare
The #6276 exposes consistency check attempts as a flag; this PR emits more histogram buckets on the
cortex-querier_storegateway_refetches_per_querymetric.Which issue(s) this PR fixes:
Fixes #
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]