Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Add retention hours to discarded metrics #15875

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

DylanGuedes
Copy link
Contributor

@DylanGuedes DylanGuedes commented Jan 21, 2025

What this PR does / why we need it:
We have no distinction between the type of data that gets discarded. On this PR I'm adding retention_hours to our discarded metrics so that we can better look at what data was discarded.

I'm also using a more performant form of retention strigifying when loading the retention string. Benchmark results:

goos: linux
goarch: amd64
pkg: github.com/grafana/loki/v3/pkg/util
cpu: AMD Ryzen 9 7900X 12-Core Processor            
BenchmarkRetentionHours/current/24h-24     	35803432	        34.14 ns/op	       2 B/op	       1 allocs/op
BenchmarkRetentionHours/current/7d-24      	33798613	        35.33 ns/op	       3 B/op	       1 allocs/op
BenchmarkRetentionHours/current/30d-24     	23425309	        50.95 ns/op	      16 B/op	       2 allocs/op
BenchmarkRetentionHours/current/365d-24    	23175356	        51.52 ns/op	      16 B/op	       2 allocs/op
BenchmarkRetentionHours/optimized/24h-24   	412592362	         2.916 ns/op	       0 B/op	       0 allocs/op
BenchmarkRetentionHours/optimized/7d-24    	89319780	        14.32 ns/op	       3 B/op	       1 allocs/op
BenchmarkRetentionHours/optimized/30d-24   	88080440	        14.60 ns/op	       3 B/op	       1 allocs/op
BenchmarkRetentionHours/optimized/365d-24  	75923084	        15.57 ns/op	       4 B/op	       1 allocs/op
PASS
ok  	github.com/grafana/loki/v3/pkg/util	11.191s

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

Checklist

  • Reviewed the CONTRIBUTING.md guide (required)
  • Documentation added
  • Tests updated
  • Title matches the required conventional commits format, see here
    • Note that Promtail is considered to be feature complete, and future development for logs collection will be in Grafana Alloy. As such, feat PRs are unlikely to be accepted unless a case can be made for the feature actually being a bug fix to existing behavior.
  • Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
  • If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

selectedRetention := o.getOverridesForUser(userID).RetentionPeriod
highestPriority := -1

if len(ls) > 0 {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this check is to help using this same function for when you want to automatically gets the Tenant retention but have no labels available

@DylanGuedes DylanGuedes force-pushed the retention-as-label-when-discarding branch from 87db09e to f269d13 Compare January 21, 2025 23:48
@DylanGuedes DylanGuedes changed the title Refactor: Add retention hours to discarded metrics refactor: Add retention hours to discarded metrics Jan 22, 2025
@DylanGuedes DylanGuedes force-pushed the retention-as-label-when-discarding branch from f269d13 to 62b8cfc Compare January 22, 2025 00:13
@DylanGuedes DylanGuedes marked this pull request as ready for review January 22, 2025 12:26
@DylanGuedes DylanGuedes requested a review from a team as a code owner January 22, 2025 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant