Add service and instance annotations to tron pods #3967
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We can currently only figure out what service/instance a log belongs to by looking at the k8s labels for the emitting pod, but label values are quite limited in length and we've got some pretty large job and/or action names, which means that the instance label for a large chunk of tronjobs ends up getting truncated.
Solution: annotations! these have a signficantly higher limit (256kb) and they can still be read by our otel collector - the only downside is that annotations cannot be used for filtering, but that's fine :)