MachineSets with Node Auto-Provisioning feature

[davivcgarcia]

About

Most of the managed Kubernetes services out there currently supports a feature commonly called as "Node Auto-Provisioning (NAP), which allows the Kubernetes to scale the worker nodes using dynamic instance types/size according the resources being requested to the cluster (PodSpecs).

• EKS: https://archive.eksworkshop.com/beginner/085_scaling_karpenter/automatic_node_provisioning/
• GKE: https://cloud.google.com/kubernetes-engine/docs/concepts/node-auto-provisioning
• AKS: https://learn.microsoft.com/en-us/azure/aks/node-autoprovision

Request

It would be nice to have similar capability on OpenShift Machine API, allowing the administrator to define a range of valid cloud instance type/sizes, and let the controller to choose the best option to accommodate cluster capacity requirements. It could not only provide Just-in-Time Node Provisioning but also Consolidation capabilities, like Karpenter provides.

https://karpenter.sh/docs/concepts/nodepools/

Example

spec:
  template:
    spec:
      requirements:
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64"]
        - key: kubernetes.io/os
          operator: In
          values: ["linux"]
        - key: karpenter.k8s.aws/instance-category
          operator: In
          values: ["c", "m", "r"]
          minValues: 2
        - key: karpenter.k8s.aws/instance-family
          operator: Exists
          minValues: 5
        - key: karpenter.k8s.aws/instance-family
          operator: In
          values: ["m5","m5d","c5","c5d","c4","r4"]
          minValues: 3
        - key: node.kubernetes.io/instance-type
          operator: Exists
          minValues: 10
        - key: karpenter.k8s.aws/instance-generation
          operator: Gt
          values: ["2"]

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[davivcgarcia]

I'm still waiting for some official feedback here. Looks like there are some experimentations around a Karpenter provisioner for Cluster API, which could be the integration point for OpenShift.

https://github.com/kubernetes-sigs/karpenter-provider-cluster-api

/remove-lifecycle stale

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale