Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring after #1481 - removing not needed env variables #1540

Open
wants to merge 35 commits into
base: main
Choose a base branch
from

Conversation

pszkamruk-splunk
Copy link
Contributor

Description:
Cleanup and refactor work to prevent code duplication related to setting Kubernetes attributes by using k8sAttributesProcessor for cluster receiver introduced by PR

Once following two PRs will be merged we can merge this one:

We need also decide what to do with resource/add_collector_k8s: link
as this is using env variables which we want to remove, but the functionality of adding additional parameters is covered by k8sattributes/metrics, so probably it is safe to remove resource/add_collector_k8s from configuration

@jvoravong
Copy link
Contributor

jvoravong commented Nov 19, 2024

For resource/add_collector_k8s:

  • Replace this with a k8sattributes processor.
  • Aim to use a single instance of the k8sattributes processor if possible. If required, add a second instance (e.g., k8sattributes/add_collector) for handling collector metrics specifically.

Functional Tests cover internal collector metrics and attributes (e.g., otelcol_exporter_send_failed_spans).

  • Check to see if you can get the functional tests to pass with using only a single k8sattributes processor instance. Add a second processor instance (named something like k8sattributes/add_collector) because the tests failed and it is necessary.

@atoulme
Copy link
Contributor

atoulme commented Dec 10, 2024

sorry, please fix conflicts

@pszkamruk-splunk pszkamruk-splunk changed the title Refactoring after #1481 - removing not needed env variables [WiP] Refactoring after #1481 - removing not needed env variables Dec 12, 2024
…dex from annotation

� Conflicts:
�	helm-charts/splunk-otel-collector/templates/config/_common.tpl
@pszkamruk-splunk pszkamruk-splunk force-pushed the removing-not-needed-env-variables branch from 556fad6 to 88f9bf8 Compare December 12, 2024 16:00
@pszkamruk-splunk
Copy link
Contributor Author

pszkamruk-splunk commented Jan 16, 2025

@jvoravong

Regarding work done here:

  1. Introduced logs and metrics attributes tests as a separate PR - [chore] Add logs and metrics k8s attributes verification tests #1610
  2. Removed environment variables from deployment-cluster-receiver.yaml
    K8S_NODE_NAME - is still there because this field is used by k8sattributes/add_collector to add k8s.node.name which is later used bysplunk-otel-collector.k8sAttributesSplunkPlatformMetrics for matching, after which additional attributes are added by k8sAttributesProcessor
  3. We need to keep initContainers.env variables (link) - these are used by /splunk-scripts/init-eks-fargate-cluster-receiver.sh
  4. updated metadata section at splunk-otel-collector.k8sAttributesSplunkPlatformMetrics - previously no attributes were added [] - this change also caused that functional test kubernetes_cluster_metrics had to be updated

Once this PR will be merged, next steps are:

  1. Unify splunk-otel-collector.k8sClusterReceiverAttributesProcessor, splunk-otel-collector.k8sAttributesProcessor, splunk-otel-collector.k8sAttributesSplunkPlatformMetrics into one k8sAttributesProcessor
  2. A change for functional tests will be needed
  3. I am also considering adding k8sattributes/add_collector and then using it for agent and clusterReceiver pipelines in similar way as it is used right now for metrics in clusterReceiver

PS: Let's merge first #1610 as I pulled it into this PR, then I will rebase and squash and we can merge this one.

@pszkamruk-splunk pszkamruk-splunk changed the title [WiP] Refactoring after #1481 - removing not needed env variables Refactoring after #1481 - removing not needed env variables Jan 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants