[Bug]: youki does not conform with OCI runtime spec

[eternalphane]

Bug Description

builder_impl::ContainerBuilderImpl::run_container calls hooks::run_hook(hooks.create_runtime(), ...) before container main process is created, while container is still in creating state. However, the spec states that

• status (string, REQUIRED) is the runtime state of the container.
  The value MAY be one of:

  • creating: the container is being created (step 2 in the lifecycle)
  • created: the runtime has finished the create operation (after step 2 in the lifecycle), and the container process has neither exited nor executed the user-specified program

and

Lifecycle

The lifecycle describes the timeline of events that happen from when a container is created to when it ceases to exist.

...
2. The container's runtime environment MUST be created according to the configuration in config.json.
...
4. The createRuntime hooks MUST be invoked by the runtime.

which indicates that create_runtime hook should only be executed when container is in created state.

---

I found this bug when trying to create a rootless container in nerdctl with youki as runtime. youki failed and complains:

error in executing command: hook command exited with non-zero exit code: 1

which is in turn caused by a fatal error in nerdctl createRuntime hook:

both state.Pid and the netNs annotation are unset

Steps to Reproduce

1. install nerdctl v2.0.0 rootless and enable debug logging in $XDG_CONFIG_HOME/nerdctl/nerdctl.toml

   debug = true
   debug_full = true

2. install youki main with dev profile

   cargo install --git https://github.com/youki-dev/youki.git youki --locked --features v1,v2,systemd,seccomp --debug
   

3. enable youki debug logging in $XDG_CONFIG_HOME/systemd/user/containerd.service.d/youki-debug.conf

   [Service]
   Environment=RUST_BACKTRACE=full
   Environment=YOUKI_LOG_LEVEL=debug
   

4. create a container with nerdctl

   nerdctl run --rm -it --runtime=youki alpine
   

5. the following error occurs (note that I modified the nerdctl source code to log the received state in createRuntime hook)

   DEBU[0000] stateDir: /run/user/1000/containerd-rootless
   DEBU[0000] RootlessKit detach-netns mode: true
   DEBU[0000] rootless parent main: executing "/usr/bin/nsenter" with [-r/ -w/home/******** --preserve-credentials -m -U -t 419558 -F nerdctl run --rm -it --runtime=youki alpine]
   DEBU[0000] failed to run [aa-exec -p nerdctl-default -- true]: "[509964] aa-exec: ERROR: profile 'nerdctl-default' does not exist\n"  error="exit status 1"
   DEBU[0000] verifying process skipped
   DEBU[0000] generated log driver: binary:///home/********/.local/bin/nerdctl?_NERDCTL_INTERNAL_LOGGING=%2Fhome%2F********%2F.local%2Fshare%2Fnerdctl%2F1935db59
   DEBU[0000] remote introspection plugin filters           filters="[type==io.containerd.snapshotter.v1, id==stargz]"
   FATA[0000] failed to create shim task: OCI runtime create failed: youki did not terminate successfully: exit status 1: time="2024-11-14T17:25:20+08:00" level=debug msg="raw state = {\"ociVersion\":\"v1.0.2\",\"id\":\"bd92ad281747c97cb5ef0d4870d21abc2b9a6ade2a740b2ad7afc89544b7047d\",\"status\":\"creating\",\"bundle\":\"/run/containerd/io.containerd.runtime.v2.task/default/bd92ad281747c97cb5ef0d4870d21abc2b9a6ade2a740b2ad7afc89544b7047d\",\"annotations\":{\"nerdctl/hostname\":\"bd92ad281747\",\"nerdctl/name\":\"alpine-bd92a\",\"nerdctl/log-uri\":\"binary:///home/********/.local/bin/nerdctl?_NERDCTL_INTERNAL_LOGGING=%2Fhome%2F********%2F.local%2Fshare%2Fnerdctl%2F1935db59\",\"nerdctl/namespace\":\"default\",\"nerdctl/ipc\":\"{\\\"mode\\\":\\\"private\\\"}\",\"nerdctl/platform\":\"linux/amd64\",\"nerdctl/state-dir\":\"/home/********/.local/share/nerdctl/1935db59/containers/default/bd92ad281747c97cb5ef0d4870d21abc2b9a6ade2a740b2ad7afc89544b7047d\",\"nerdctl/auto-remove\":\"true\",\"nerdctl/networks\":\"[\\\"bridge\\\"]\",\"nerdctl/extraHosts\":\"null\"},\"useSystemd\":true,\"cleanUpIntelRdtSubdirectory\":null}"
   time="2024-11-14T17:25:20+08:00" level=fatal msg="both state.Pid and the netNs annotation are unset"
   error in executing command: hook command exited with non-zero exit code: 1
   
   Stack backtrace:
      0: <unknown>
      1: <unknown>
      2: <unknown>
      3: <unknown>
      4: <unknown>
      5: <unknown>
      6: <unknown>
      7: <unknown>
      8: <unknown>
      9: <unknown>
     10: <unknown>
     11: __libc_start_main
     12: <unknown>
   Error: hook command exited with non-zero exit code: 1
   
   Stack backtrace:
      0: <unknown>
      1: <unknown>
      2: <unknown>
      3: <unknown>
      4: <unknown>
      5: <unknown>
      6: <unknown>
      7: <unknown>
      8: <unknown>
      9: <unknown>
     10: <unknown>
     11: __libc_start_main
     12: <unknown>
   : unknown
   

   Note that state.status field is creating, and state.pid field is missing

Expectation

youki should call hooks::run_hook(hooks.create_runtime(), ...)) after process::container_main_process::container_main_process(...), with state.status == "created" and state.pid == init_pid

System and Setup Info

Version           0.4.1
Commit            VERGEN_IDEMPOTENT_OUTPUT
Kernel-Release    6.11.5-amd64
Kernel-Version    #1 SMP PREEMPT_DYNAMIC Debian 6.11.5-1 (2024-10-27)
Architecture      x86_64
Operating System  Debian GNU/Linux trixie/sid
Cores             4
Total Memory      15331
Cgroup setup      unified
Cgroup mounts
  unified         /sys/fs/cgroup
CGroup v2 controllers
  cpu             attached
  cpuset          attached
  hugetlb         attached
  io              attached
  memory          attached
  pids            attached
  device          attached
Namespaces        enabled
  mount           enabled
  uts             enabled
  ipc             enabled
  user            enabled
  pid             enabled
  network         enabled
  cgroup          enabled
Capabilities
CAP_BPF           available
CAP_PERFMON       available
CAP_CHECKPOINT_RESTORE available

Additional Context

No response