Templated Share Subdirectory

[MiddleMan5]

Is your feature request related to a problem?/Why is this needed

We have a number of SMB NAS systems that we use as cluster-wide, general purpose storage, but provisioning sub-directories for each application is time-consuming and error prone and requires a human admin to configure.

As of now, we have to define a csi-smb-provisioned StorageClass per application deployment just to reliably control the directory that an application stores it's data in which again requires another additional cluster-admin step.

Describe the solution you'd like in detail
Ideally, a StorageClass could define a "subDir" template string that allows the admin to control how a new PVC is allocated a SMB share.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: smb-auto
provisioner: smb.csi.k8s.io
parameters:
  source: "//mynas.my.domain/storage/root"
  subDir: "smb-auto-${pvc.annotations['app.kubernetes.io/instance']}"
  csi.storage.k8s.io/node-stage-secret-name: "smbcreds"
  csi.storage.k8s.io/node-stage-secret-namespace: "storage"

Given the above as an example of suggested syntax, The following additional use cases should be taken into consideration:

A unique share name per pvc:
Given: subDir: "smb-auto-${pvc.metadata.name}"
Result: //mynas.my.domain/storage/root/smb-auto-pvc-95f1d979-9cb2-4a84-90ed-5d25cebeccc4

Namespace-specific shared storage:
Given: subDir: "smb-auto-${pvc.metadata.namespace}"
Result: //mynas.my.domain/storage/root/smb-auto-production

A shared storage share that multiple pods of the same deployment share
Given: subDir: "smb-auto-${pvc.annotations['app.kubernetes.io/instance']}"
Result: //mynas.my.domain/storage/root/smb-auto-nginx-frontend

Describe alternatives you've considered
This feature request appears to be related to the discussion under issue #428

Additional context
Kubernetes 1.23

[andyzhangx]

I think there is a way to implement this in smb csi driver for the first two, we should set --extra-create-metadata=true in csi-provisioner, and then replace ${pvc.metadata.name} with csi.storage.k8s.io/pvc/name value, replace ${pvc.metadata.namespace} with csi.storage.k8s.io/pvc/namespace passed in CreateVolume.parameters, and for the third requirement(${pvc.annotations['app.kubernetes.io/instance']}"), since it's not passed by csi-provisioner, so it's not supported yet.

[MiddleMan5]

Is there any information that is inferred from the final path name that is used in deletion that would need to be stored if the PVC is deleted, or is that information contained in the delete event?

[andyzhangx]

Is there any information that is inferred from the final path name that is used in deletion that would need to be stored if the PVC is deleted, or is that information contained in the delete event?

@MiddleMan5 the final subDir should be part of volumeid, so when deletion happen, the driver would know which subDir should be deleted. following are the testing logs, volume id format is like source#subdir#uuid

[pod/csi-smb-controller-774469f654-cpckx/smb] I0613 07:56:39.754001       1 utils.go:76] GRPC call: /csi.v1.Controller/DeleteVolume
[pod/csi-smb-controller-774469f654-cpckx/smb] I0613 07:56:39.754024       1 utils.go:77] GRPC request: {"secrets":"***stripped***","volume_id":"smb-server.default.svc.cluster.local/share#subDirectory#pvc-bbcaf000-0fa7-4bfa-b2af-716e0da5b97f"}
[pod/csi-smb-controller-774469f654-cpckx/smb] I0613 07:56:39.754078       1 controllerserver.go:136] DeleteVolume: found mountOptions(dir_mode=0777,file_mode=0777,uid=0,gid=0,mfsymlinks) for volume(smb-server.default.svc.cluster.local/share#subDirectory#pvc-bbcaf000-0fa7-4bfa-b2af-716e0da5b97f)
[pod/csi-smb-controller-774469f654-cpckx/smb] I0613 07:56:39.754086       1 controllerserver.go:255] internally mounting //smb-server.default.svc.cluster.local/share at /tmp/pvc-bbcaf000-0fa7-4bfa-b2af-716e0da5b97f
[pod/csi-smb-controller-774469f654-cpckx/smb] I0613 07:56:39.754180       1 nodeserver.go:194] NodeStageVolume: targetPath(/tmp/pvc-bbcaf000-0fa7-4bfa-b2af-716e0da5b97f) volumeID(smb-server.default.svc.cluster.local/share#subDirectory#pvc-bbcaf000-0fa7-4bfa-b2af-716e0da5b97f) context(map[source://smb-server.default.svc.cluster.local/share]) mountflags([dir_mode=0777,file_mode=0777,uid=0,gid=0,mfsymlinks]) mountOptions([dir_mode=0777,file_mode=0777,uid=0,gid=0,mfsymlinks])
[pod/csi-smb-controller-774469f654-cpckx/smb] I0613 07:56:39.754431       1 mount_linux.go:183] Mounting cmd (mount) with arguments (-t cifs -o dir_mode=0777,file_mode=0777,uid=0,gid=0,mfsymlinks,<masked> //smb-server.default.svc.cluster.local/share /tmp/pvc-bbcaf000-0fa7-4bfa-b2af-716e0da5b97f)
[pod/csi-smb-controller-774469f654-cpckx/smb] I0613 07:56:39.815035       1 nodeserver.go:223] volume(smb-server.default.svc.cluster.

[andyzhangx]

PR #503 works with "subDir":"subDirectory-${pvc.metadata.name}", I will refine that PR a little and then merge,
"subDir":"subDirectory-${pvc.metadata.name}" would be converted to "subDir":"subDirectory-pvc-w7sbh" in below test:

[pod/csi-smb-controller-7584fbf655-g28tt/smb] I0617 15:22:39.115777       1 utils.go:76] GRPC call: /csi.v1.Controller/CreateVolume
[pod/csi-smb-controller-7584fbf655-g28tt/smb] I0617 15:22:39.115804       1 utils.go:77] GRPC request: {"capacity_range":{"required_bytes":107374182400},"name":"pvc-a50b373f-6a11-42b5-a838-ff4b949229fb","parameters":{"csi.storage.k8s.io/pv/name":"pvc-a50b373f-6a11-42b5-a838-ff4b949229fb","csi.storage.k8s.io/pvc/name":"pvc-w7sbh","csi.storage.k8s.io/pvc/namespace":"smb-1737","source":"//smb-server.default.svc.cluster.local/share","subDir":"subDirectory-${pvc.metadata.name}"},"secrets":"***stripped***","volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
[pod/csi-smb-controller-7584fbf655-g28tt/smb] I0617 15:22:39.115929       1 controllerserver.go:84] create subdirectory(subDirectory-pvc-w7sbh) if not exists
[pod/csi-smb-controller-7584fbf655-g28tt/smb] I0617 15:22:39.115939       1 controllerserver.go:255] internally mounting //smb-server.default.svc.cluster.local/share at /tmp/pvc-a50b373f-6a11-42b5-a838-ff4b949229fb
[pod/csi-smb-controller-7584fbf655-g28tt/smb] I0617 15:22:39.116042       1 nodeserver.go:200] NodeStageVolume: targetPath(/tmp/pvc-a50b373f-6a11-42b5-a838-ff4b949229fb) volumeID(smb-server.default.svc.cluster.local/share#subDirectory-pvc-w7sbh#pvc-a50b373f-6a11-42b5-a838-ff4b949229fb) context(map[source://smb-server.default.svc.cluster.local/share]) mountflags([]) mountOptions([])
[pod/csi-smb-controller-7584fbf655-g28tt/smb] I0617 15:22:39.116330       1 mount_linux.go:183] Mounting cmd (mount) with arguments (-t cifs -o <masked> //smb-server.default.svc.cluster.local/share /tmp/pvc-a50b373f-6a11-42b5-a838-ff4b949229fb)
[pod/csi-smb-controller-7584fbf655-g28tt/smb] I0617 15:22:39.146537       1 nodeserver.go:234] volume(smb-server.default.svc.cluster.local/share#subDirectory-pvc-w7sbh#pvc-a50b373f-6a11-42b5-a838-ff4b949229fb) mount "//smb-server.default.svc.cluster.local/share" on "/tmp/pvc-a50b373f-6a11-42b5-a838-ff4b949229fb" succeeded
[pod/csi-smb-controller-7584fbf655-g28tt/smb] I0617 15:22:39.151722       1 controllerserver.go:273] internally unmounting /tmp/pvc-a50b373f-6a11-42b5-a838-ff4b949229fb
[pod/csi-smb-controller-7584fbf655-g28tt/smb] I0617 15:22:39.151755       1 nodeserver.go:256] NodeUnstageVolume: CleanupMountPoint on /tmp/pvc-a50b373f-6a11-42b5-a838-ff4b949229fb with volume smb-server.default.svc.cluster.local/share#subDirectory-pvc-w7sbh#pvc-a50b373f-6a11-42b5-a838-ff4b949229fb
[pod/csi-smb-controller-7584fbf655-g28tt/smb] I0617 15:22:39.151774       1 mount_linux.go:294] Unmounting /tmp/pvc-a50b373f-6a11-42b5-a838-ff4b949229fb
[pod/csi-smb-controller-7584fbf655-g28tt/smb] W0617 15:22:39.160546       1 mount_helper_common.go:133] Warning: "/tmp/pvc-a50b373f-6a11-42b5-a838-ff4b949229fb" is not a mountpoint, deleting
[pod/csi-smb-controller-7584fbf655-g28tt/smb] I0617 15:22:39.160621       1 nodeserver.go:261] NodeUnstageVolume: unmount volume smb-server.default.svc.cluster.local/share#subDirectory-pvc-w7sbh#pvc-a50b373f-6a11-42b5-a838-ff4b949229fb on /tmp/pvc-a50b373f-6a11-42b5-a838-ff4b949229fb successfully
[pod/csi-smb-controller-7584fbf655-g28tt/smb] I0617 15:22:39.160632       1 utils.go:83] GRPC response: {"volume":{"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a50b373f-6a11-42b5-a838-ff4b949229fb","csi.storage.k8s.io/pvc/name":"pvc-w7sbh","csi.storage.k8s.io/pvc/namespace":"smb-1737","source":"//smb-server.default.svc.cluster.local/share","subDir":"subDirectory-pvc-w7sbh"},"volume_id":"smb-server.default.svc.cluster.local/share#subDirectory-pvc-w7sbh#pvc-a50b373f-6a11-42b5-a838-ff4b949229fb"}}