Latest release is not backwards compatible

[CecileRobertMichon]

What happened:

Jobsets created with older releases cannot be modified or delete themselves after they finish.

Spec.network is immutable via CEL but the jobset defaulting webhook now defaults network.publishNotReadyAddresses which means:

• it is not possible to update jobsets created with older versions
• jobsets created with older versions do not get Garbage Collected because the finalizer is never removed, even after all children have been deleted

Making any changes to an older jobset without publishNotReadyAddresses set will result in the defaulting webhook defaulting it to true, which then gets rejected as an update by CEL because "Value is immutable".

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

• Create a jobset with version v0.5.0
• Update the version of jobset to v.7+
• Try to change any property of the jobset and observe that it fails
• When the jobset finishes, it cannot delete itself because the finalizer cannot be removed

Anything else we need to know?:

Environment:

• Kubernetes version (use kubectl version):
• JobSet version (use git describe --tags --dirty --always):
• Cloud provider or hardware configuration:
• Install tools:
• Others:

[kannon92]

How do you do an upgrade?

[kannon92]

I am having some trouble reproducing this. I am using kind with k8s 1.31.

I install jobset 0.5.0 on kind with k8s version 1.31 and apply the following jobset:

apiVersion: jobset.x-k8s.io/v1alpha2
kind: JobSet
metadata:
  name: paralleljobs
spec:
  suspend: true
  replicatedJobs:
  - name: workers
    template:
      spec:
        parallelism: 4
        completions: 4
        backoffLimit: 0
        template:
          spec:
            containers:
            - name: sleep
              image: busybox
              command: 
                - sleep
              args:
                - 100s
  - name: driver
    template:
      spec:
        parallelism: 1
        completions: 1
        backoffLimit: 0
        template:
          spec:
            containers:
            - name: sleep
              image: busybox
              command: 
                - sleep
              args:
                - 100s

I then "upgrade" Jobset via

kubectl apply --server-side -f https://github.com/kubernetes-sigs/jobset/releases/download/v0.7.3/manifests.yaml

I am able to do kubectl edit jobset and change suspend -> false. The update goes through without issue and the JobSet is triggered.