bug: gitaly cgroups init customization causing crashloop

[JoeHCQ1]

10 days ago a deploy to dev resulted in a crashlooping gitaly. This worked when I merged my PR, but doesn't now.

[JoeHCQ1]

Tested it - the issue is that the security context isn't getting set correctly.

Even with this in the bundle:

            - path: gitlab.gitaly.cgroups.initContainer.securityContext # These should be the default https://gitlab.com/gitlab-org/charts/gitlab/-/blob/master/charts/gitlab/charts/gitaly/values.yaml?ref_type=heads#L56
              value:                                                    # but are overridden somewhere to 1000 so setting them back to 0 here.
                runAsUser: 0
                runAsGroup: 0
                privileged: true  # privileged needs added, was not in upstream helm
                readOnlyRootFilesystem: false
                allowPrivilegeEscalation: true
                runAsNonRoot: false

I only get this in the statefulset:

                runAsUser: 0
                runAsGroup: 0

If I manually edit that back into the statefulset, the gitaly init container works. Pepr labels show no changes made to the pod.

This line would need figured out: https://gitlab.com/gitlab-org/charts/gitlab/-/blob/master/charts/gitlab/charts/gitaly/templates/_statefulset_spec.yaml?ref_type=heads#L63

[JoeHCQ1]

This is blocked by https://gitlab.com/gitlab-org/gitlab/-/issues/507883. The gitlab helm chart isn't written such that we can get past this.

[JoeHCQ1]

This includes some related work: defenseunicorns/uds-package-gitlab#246

[JoeHCQ1]

I believe this was resolvable by overloading the values.yaml with an input that bypassed the upstream chart limitation.

[JoeHCQ1]

This yaml snippet in the gitlab YAML would appear to be how I may have had it working:

gitlab:
  gitaly:
    cgroups: # .Values.cgroups.initContainer.securityContext
      enabled: true
      initContainer:
        securityContext:
          runAsUser: |
            0
              privileged: true
          runAsGroup: 0

[JoeHCQ1]

The exact form of this was very sensitive. That ^ worked iirc, any variation, including ways of substituting it in via UDS Variables did not.

[JoeHCQ1]

Looks like I also needed to edit the statefulset for gitlab's helper chart like so:

@@ -60,7 +60,10 @@ spec:
         - name: init-cgroups
           image: {{ include "gitlab.configure.image" (dict "root" $ "image" .Values.cgroups.initContainer.image) | quote }}
           {{- include "gitlab.image.pullPolicy" $initCgroupsImageCfg | indent 10 }}
-          {{- include "gitlab.podSecurityContextRoot" .Values.cgroups.initContainer.securityContext | nindent 10 }}
+          {{- if .Values.cgroups.initContainer.securityContext }}
+          securityContext:
+            {{- toYaml .Values.cgroups.initContainer.securityContext | nindent 12 }}
+          {{- end }}