customConfigSecret breaks the NodeAgent

[bmack94]

Describe the bug:
When a customConfigSecret is defined in the nodeAgent it is mounted on a different directory (/fluent-bit/etc/ instead of /fluent-bit/conf_operator)

logging-operator/pkg/resources/nodeagent/daemonset.go

Line 155 in 82d2fd6

MountPath: "/fluent-bit/etc/",

But the the Command is not updated and stays at /fluent-bit/conf_operator/

logging-operator/pkg/resources/nodeagent/nodeagent.go

Line 67 in 82d2fd6

Command: []string{"/fluent-bit/bin/fluent-bit", "-c", "/fluent-bit/conf_operator/fluent-bit.conf"},

This also removes all default config from the fluent-bit image like the parsers which where stored in that directory.
The fluentbitspec we used in the past did only overwrite the SecretVolumeName:

logging-operator/pkg/resources/fluentbit/daemonset.go

Line 260 in 82d2fd6

SecretName: r.fluentbitSpec.CustomConfigSecret,

Expected behaviour:
When a customConfigSecret is defined the SecretVolumeName should be overwritten instead of mounted over the upstream directory.

Steps to reproduce the bug:
Apply the helm values from below

Workaround:
For those who also want to use the nodeAgent i got it working with the following:

    nodeAgents:
    - name: linux-agent
      nodeAgentFluentbit:
        daemonSet:
          spec:
            template:
              spec:
                containers:
                - command:
                  - /fluent-bit/bin/fluent-bit
                  - -c
                  - /fluent-bit/etc-operator/fluent-bit.conf
                  name: fluent-bit
                  volumeMounts:
                  - mountPath: /fluent-bit/etc-operator
                    name: custom-config
                tolerations:
                - effect: NoSchedule
                  key: node-role.kubernetes.io/control-plane
                  operator: Exists
                volumes:
                - name: custom-config
                  secret:
                    defaultMode: 420
                    secretName: fluentbit-config

Environment details:

• Kubernetes version: v1.28.8
• Cloud-provider/provisioner: RKE2
• logging-operator version (e.g. 2.1.1): 5.0.1
• Install method: Helm
• Logs from the misbehaving component (and any other relevant logs):

[2025/01/15 11:36:37] [error] could not open configuration file, aborting.

• Resource definition (possibly in YAML format) that caused the issue, without sensitive data:
  Helm Values (relevant part)

logging:
  enabled: true
  nodeAgents:
    - name: linux-agent
      profile: linux
      nodeAgentFluentbit:
        customConfigSecret: fluentbit-config
        metrics:
          serviceMonitor: true
          prometheusRules: true
        daemonSet:
          spec:
            template:
              spec:
                tolerations:
                  - effect: NoSchedule
                    key: node-role.kubernetes.io/control-plane
                    operator: Exists

DaemonSet:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: logging-operator-linux-agent-fluentbit
  namespace: kube-logging
spec:
  selector:
    matchLabels:
      app.kubernetes.io/instance: linux-agent
      app.kubernetes.io/managed-by: logging-operator
      app.kubernetes.io/name: fluentbit
  template:
    metadata:
      creationTimestamp: null
      labels:
        app.kubernetes.io/instance: linux-agent
        app.kubernetes.io/managed-by: logging-operator
        app.kubernetes.io/name: fluentbit
    spec:
      containers:
      - command:
        - /fluent-bit/bin/fluent-bit
        - -c
        - /fluent-bit/conf_operator/fluent-bit.conf
        image: fluent/fluent-bit:3.1.8
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /api/v1/metrics/prometheus
            port: 2020
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: fluent-bit
        ports:
        - containerPort: 2020
          name: monitor
          protocol: TCP
        resources:
          limits:
            cpu: 200m
            memory: 100M
          requests:
            cpu: 100m
            memory: 50M
        securityContext: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /var/lib/docker/containers
          name: containerspath
        - mountPath: /var/log
          name: varlogspath
        - mountPath: /fluent-bit/etc/
          name: config
        - mountPath: /tail-db
          name: positiondb
        - mountPath: /buffers
          name: buffers
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: logging-operator-linux-agent-fluentbit
      serviceAccountName: logging-operator-linux-agent-fluentbit
      terminationGracePeriodSeconds: 30
      tolerations:
      - effect: NoSchedule
        key: node-role.kubernetes.io/control-plane
        operator: Exists
      volumes:
      - hostPath:
          path: /var/lib/docker/containers
          type: ""
        name: containerspath
      - hostPath:
          path: /var/log
          type: ""
        name: varlogspath
      - name: config
        secret:
          defaultMode: 420
          secretName: fluentbit-config
      - emptyDir: {}
        name: positiondb
      - emptyDir: {}
        name: buffers
  updateStrategy:
    rollingUpdate:
      maxSurge: 0
      maxUnavailable: 1
    type: RollingUpdate

/kind bug