Hi @cedricve,

Could you please provide the value of the hami.io/node-nvidia-register key in the node annotations after modifying the ConfigMap and restarting the hami-device-plugin? This will help us diagnose the issue.

Thanks!

Hi @Nimbus318

the value of key is "hami.io/node-nvidia-register: 'GPU-3fe7a6f9-8bfb-3edd-daa9-610a511adf22,10,24564,100,NVIDIA-NVIDIA"

The second field, 10, represents the maximum number of splits for this GPU. I would like to confirm the current issue: After setting deviceSplitCount in the ConfigMap to a value greater than 10 and restarting the hami-device-plugin, the second field in the hami.io/node-nvidia-register annotation remains 10. Is that correct?

That is correct. @mrabiaa can you also confirm this.

Hello @Nimbus318 I just edit the config map and restart the pods of Hami and still the same issue
$ kubectl get node -o jsonpath='{.metadata.annotations.hami.io/node-nvidia-register}'
GPU-c1f6532b-9a21-0a33-3f52-b7a173c66ee8,10,24564,100,NVIDIA-NVIDIA GeForce RTX 4090,0,true

$ kubectl get cm -n kube-system hami-device-plugin -o yaml
apiVersion: v1
data:
config.json: |
{
"nodeconfig": [
{
"name": "m5-cloudinfra-online02",
"devicememoryscaling": 1.8,
"devicesplitcount": 20,
"migstrategy":"none",
"filterdevices": {
"uuid": [],
"index": []
}
}
]
}
kind: ConfigMap
metadata:
annotations:
meta.helm.sh/release-name: hami
meta.helm.sh/release-namespace: kube-system
creationTimestamp: "2024-11-21T02:00:45Z"
labels:
app.kubernetes.io/component: hami-device-plugin
app.kubernetes.io/instance: hami
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: hami
app.kubernetes.io/version: 2.4.1
helm.sh/chart: hami-2.4.1
name: hami-device-plugin
namespace: kube-system
resourceVersion: "34659059"
uid: ea7385b9-8b1c-4b82-a3ce-dcf8328647f1

@mrabiaa @cedricve Thank you for your investigation. I can confirm that the issue is related to the node name configuration in the ConfigMap. The name field in the nodeconfig section needs to match exactly with your target node's name (the node where you want to set devicesplitcount: 20).
To fix this, update your ConfigMap with the correct node name:

"nodeconfig": [
  {
    "name": "<your-actual-node-name>",  # Replace with your node name
    "devicememoryscaling": 1.8,
    "devicesplitcount": 20,
    "migstrategy": "none",
    "filterdevices": {
      "uuid": [],
      "index": []
    }
  }
]

After updating the ConfigMap with the correct node name, remember to restart the HAMi device plugin pods for the changes to take effect.

@Nimbus318 awesome , solved
thank you

Great thank you!