Kubernetes runAsNonRoot incompatibility with run-user: _daemon_

[berkayoz]

Bug Description

Some deployments are setting runAsNonRoot: true for the pod security context which prevents a container from running as root. To possibly address this we've set run-user: _daemon_ however Kubernetes checks uid/gid numerically and complains about the _daemon_ user.

You can find the Kubernetes implementation here

As a work around we have to set securityContext.runAsUser=584792 manually on manifests which hurts the drop-in image replacement story. Could we possibly set numeric UID as the OCI user by default?

Your help is much appreciated, many thanks!

To Reproduce

Deploy a Pod that uses a rock with the securityContext set as so

spec:
  securityContext:
    runAsNonRoot: true

You can also check out our cert-manager tests to verify it in a real world example.

Environment

Ubuntu 24.04

rockcraft.yaml

name: cert-manager-controller
summary: ROCK for the cert-manager-controller Project.
description: |
  This ROCK is a drop-in replacement for the autoscaling/cert-manager-controller image.
version: "1.12.2"
license: Apache-2.0

base: bare
build-base: [email protected]
platforms:
  amd64:
  arm64:

run-user: _daemon_
entrypoint-service: cert-manager-controller
services:
  cert-manager-controller:
    override: replace
    summary: "cert-manager-controller service"
    startup: enabled
    command: "/controller-linux [ -h ]"
    on-failure: shutdown

parts:
  cert-manager-controller:
    plugin: nil
    source: https://github.com/cert-manager/cert-manager.git
    source-type: git
    source-tag: v1.12.2
    source-depth: 1
    build-snaps:
      - jq
      - go/1.22/stable
    override-build: |
      # CTR=echo is hacky way of passing docker check not required for build
      make CTR=echo _bin/server/controller-linux-${CRAFT_PLATFORM}
      cp _bin/server/controller-linux-${CRAFT_PLATFORM} ${CRAFT_PART_INSTALL}/controller-linux
    prime:
      - controller-linux

Relevant log output

Error: container has runAsNonRoot and image has non-numeric user (_daemon_), cannot verify user is non-root

[kimwnasptd]

I took a slightly deeper look on this issue. It looks like the root cause is that:

1. A rock's OCI Image Configuration will set .config.User: "_daemon_"
2. Looking at containerd (although other CRIs might be different)
   1. it parses the. .config.User when implementing the ImageStatus CRI API [1] [2]
   2. in rock's case, containerd returns back to kubelet just _daemon_
3. kubelet will then bump into the line mentioned in the issue and complain [1] [2] [3]

I looked at another popular Kubeflow image, which has a non-root user kubeflownotebookswg/jupyter-scipy:v1.9.0 and after inspecting its OCI Image Configuration JSON file, I see that in this case it's like this:

$ cat 5f3bb424e9ed386124a1762cbdf46bfe4e2f96f9b2fb3ca82474877b2d06bc75.json | jq ".config.User"
"1000"

Also in the above image, if I execute a binary it will indeed be a non-root user:

docker run -ti --entrypoint bash kubeflownotebookswg/jupyter-scipy:v1.9.0
$ id -u $(whoami)
1000

So from the above it looks like a possible fix would be to set in the OCI Image Configuration the UID directly, in this case 584792, and not _daemon_.

[kimwnasptd]

What's not clear to me though is why kubelet decides to have this logic:

1. If it got a UID, it will compare if it's 0 (root)
2. But if it got a username, and no UID, then it will complain it can't deduce if the user is root or not

Can a user that is not named root actually have UID 0?