To obtain the "Allocated resources:" from the output of kubectl describe node <node-name>, what query should be used?

[githubeto]

I believe that the most accurate value for determining whether a Pod can be scheduled is the Requests section under
"Allocated resources:" obtained from kubectl describe node .

This is because if these values are close to 100% (for example, 99%), attempting to schedule a Pod will result in a resource shortage, making it impossible to schedule.

So, how can we obtain this value using a Prometheus query in Grafana?

The reason I am asking this question is that while the query
kube_node_status_allocatable{resource="memory"}
can be used to obtain the total capacity of the node, the sum of the memory requests of currently scheduled Pods obtained with
sum by (node) (kube_pod_container_resource_requests{resource="memory"})
clearly exceeds the node's capacity. Each of my nodes has 16Gi of memory, but this query returns a total that exceeds 16Gi.

• sum by (node) (kube_pod_container_resource_requests{resource="memory"})
  

18.2Gi ...? (Worker node is 16Gi memory spec.)

[k8s-ci-robot]

This issue is currently awaiting triage.

If kube-state-metrics contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[xiaozongyang]

Hi @githubeto, we meet same issure.

Since the metric kube_pod_container_resource_requests exposing all requests of pods including Completed|Failed pods, value of sum(kube_pod_container_resource_requests) will exceeds actual max memory.

IMO, we should add new metric named kube_node_resource_requests and kube_node_resource_limits exposing value from kubectl describe node

[xiaozongyang]

The other metrics kube_node_.* are implemented in node.go. It depents on the struct v1.Node which doesn't have any field about resource requests and limits.

It seems that we should import other struct to expose kube_node_resource_requests