feat: add GCP ccm

[frezbo]

Update docs to reflect deploying GCP Cloud Control Manager (CCM)

Signed-off-by: Noel Georgi [email protected]

---

This change is 

[frezbo]

/approve

[frezbo]

/m -ff

[smira]

/m -ff

you need first some reviews on the PR

[smira]

@rsmitty could be the best resource to review this one, I'm not familiar enough with this

[nberlee]

@frezbo
You may also want to add the RBAC for the ILB to be able to write to events: (you will see an error if you create a loadbalancer with an internal ip) see https://raw.githubusercontent.com/on2itsecurity/secure-k8s/main/cluster/manifests/gce-provider-extra-rbac.yaml

Consider adding --port: 0 to the parameters (see warning in log)
And add a default gce.conf as this is where the GCP module mostly gets its config from.. .The paramaters in the cloud controller are limited...

[frezbo]

@nberlee Thanks for the feedback.

• I'll test out the internal load balancer and update if necessary.
• --port=0 is not set as the apiserver-authentication-reader role binding is created. It;s a common role binding with ccm's
• Do you have an example snippet of gce.conf, and what parameters are usually in there?. That would help in improving the docs

[nberlee]

All possible settings are here: https://github.com/kubernetes/cloud-provider-gcp/blob/master/providers/gce/gce.go#L183
my gce.conf looks something like this, although, it really depends on your GCP environment if you want to have some of these features to be autodetected. The last two are the most important I think because that makes it behave way differently...

data:
  gce.conf: |-
    [global]
    project-id = my-prod-id
    network-project-id = my-prod-id
    network-name = vpc-workers-cluster2
    subnetwork-name = sn-vips-cluster2
    node-instance-prefix = vm-worker-node
    node-tags = talos-workers-cluster2
    multizone = true
    regional = true

I must have been wrong about --port=0 then I thought this was the nonsecure CCM api port... just like kube-apiserver/scheduler/controler-manager has this option

[frezbo]

Linked issues:

• Incorrect URL format for load balancers kubernetes/cloud-provider-gcp#286
• Nodes network-unavailable taint not removed after installing ccm kubernetes/cloud-provider-gcp#291
• There are no official published images for CCM kubernetes/cloud-provider-gcp#289

[sergelogvinov]

cloud-provider-gcp does not have many features than gce-ccm.

If any one wants to run ccm without allocate-node-cidrs -- here fix sergelogvinov/cloud-provider-gcp@49216c9 (good for multi cloud setups)

in my case, I use only that params, (csi/autoscale work fine with that ccm)

          command:
            - /cloud-controller-manager
          args:
            - --bind-address=127.0.0.1
            - --cloud-provider=gce
            - --cloud-config=/etc/gce/gce.conf
            - --allocate-node-cidrs=false
            - --cluster-cidr=10.64.0.0/12
            - --controllers=cloud-node,cloud-node-lifecycle
            - --port=0
            - --use-service-account-credentials
            - -v=2

  gce.conf: |
    [global]
    project-id = 
    network-name =

[smira]

not an expert, but this looks much more fun to me

[frezbo]

/m -ff