ImagePullPolicy set to Never even if skaffold doesn't load the image

[timebertt]

In my project, I use skaffold to deploy off-the-shelf components (alongside my own components).
For these components, skaffold doesn't build the images and hence doesn't load them into a local kind cluster.
With #9495, skaffold sets imagePullPolicy=Never even if it doesn't load the image resulting in ErrImageNeverPull, e.g.:

Waiting for deployments to stabilize...
 - cert-manager:deployment/cert-manager: ErrImageNeverPull: Container image "quay.io/jetstack/cert-manager-controller:v1.16.3" is not present with pull policy of Never

Steps to reproduce:

1. Prepare the following files:

skaffold.yaml

apiVersion: skaffold/v4beta10
kind: Config
metadata:
  name: cert-manager
manifests:
  kustomize:
    paths:
      - .
deploy:
  kubectl: {}

kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- https://github.com/cert-manager/cert-manager/releases/download/v1.16.3/cert-manager.yaml

2. Create a kind cluster:

kind create cluster

3. Run skaffold:

$ skaffold run
No tags generated
Starting test...
Starting deploy...
Loading images into kind cluster nodes...
Images loaded in 167ns
 - namespace/cert-manager created
 - deployment.apps/cert-manager created
...

Waiting for deployments to stabilize...
 - cert-manager:deployment/cert-manager: ErrImageNeverPull: Container image "quay.io/jetstack/cert-manager-controller:v1.16.3" is not present with pull policy of Never
...

Also see timebertt/kubernetes-controller-sharding#437 as a real-world example.

[timebertt]

cc @lucasrod16

[tom00]

It seems it's been caused by this feature: #9495.

The issue is that Skaffold is replacing the ImagePullPolicy in all pods it's deploying, regardless of whether an image was initially built locally or not.

For for the use case when a local application is deployed together with a couple of backing services that their image should be pulled from a remote registry, it leads to ErrImageNeverPull error.