kubernetes-sigs · dabradley · Mar 15, 2025 · Feb 3, 2025 · Feb 12, 2024
diff --git a/.golangci.yaml b/.golangci.yaml
@@ -54,4 +54,4 @@ linters-settings:
   unparam:
     check-exported: true
 run:
-  timeout: 30m0s
+  timeout: 30m
diff --git a/deploy/csi-azurelustre-controller.yaml b/deploy/csi-azurelustre-controller.yaml
@@ -36,7 +36,7 @@ spec:
             - "-v=2"
             - "--csi-address=$(ADDRESS)"
             - "--leader-election"
-            - "--timeout=60s"
+            - "--timeout=15m"
             - "--extra-create-metadata=true"
           env:
             - name: ADDRESS
@@ -73,6 +73,7 @@ spec:
           args:
             - "-v=5"
             - "--endpoint=$(CSI_ENDPOINT)"
+            - "--enable-azurelustre-mock-dyn-prov=false"
           ports:
             - containerPort: 29762
               name: healthz

diff --git a/docs/driver-parameters.md b/docs/driver-parameters.md
@@ -0,0 +1,55 @@
+# Driver Parameters
+
+These are the parameters to be passed into the custom StorageClass that users must create to use the CSI driver.
+
+For more information, see the [Azure Managed Lustre Filesystem (AMLFS) service documentation](https://learn.microsoft.com/en-us/azure/azure-managed-lustre/) and the [AMLFS CSI documentation](https://learn.microsoft.com/en-us/azure/azure-managed-lustre/use-csi-driver-kubernetes).
+
+## Dynamic Provisioning (Create an AMLFS Cluster through AKS)
+
+### Permissions
+The kubelet identity attached to the cluster will require the following permission actions (at the Subscription scope):
+
+```
+Microsoft.Network/virtualNetworks/subnets/read
+Microsoft.Network/virtualNetworks/subnets/join/action
+Microsoft.StorageCache/getRequiredAmlFSSubnetsSize/action
+Microsoft.StorageCache/checkAmlFSSubnets/action
+Microsoft.StorageCache/amlFilesystems/read
+Microsoft.StorageCache/amlFilesystems/write
+Microsoft.StorageCache/amlFilesystems/delete
+```
+
+Alternatively, users can grant the identity the following broader roles:
+
+- Reader permissions the Subscription scope
+- Contributor permissions at the Resource Group scope (the Resource Group `resource-group-name` parameter of the StorageClass)
+- Network Contributor at the Virtual Network scope (the Virtual Network specified in the `vnet-name` parameter of the StorageClass) instead of the individual permissions.
+
+If using the `identities` parameter, users will also need to grant Managed Identity Operator role on the identity being assigned or the following permission action:
+```
+Microsoft.ManagedIdentity/userAssignedIdentities/assign/action
+```
+
+### Parameters
+
+Name | Meaning | Available Value | Mandatory | Default value
+--- | --- | --- | --- | ---
+sku-name | SKU name for the Azure Managed Lustre file system. The SKU determines the throughput of the AMLFS cluster. | The SKU value must be one of the following: `AMLFS-Durable-Premium-40`, `AMLFS-Durable-Premium-125`, `AMLFS-Durable-Premium-250`, `AMLFS-Durable-Premium-500`. | Yes | This value must be provided.
+zones | The availability zone where your resource will be created. For the best performance, locate your AMLFS cluster in the same region and availability zone that houses your AKS cluster and other compute clients. | The zone must be a single value e.g., `"1"`, `"2"`, or `"3"`. | Yes | This value must be provided.
+maintenance-day-of-week | The day of the week for maintenance to be performed on the AMLFS cluster. | `Sunday`, `Monday`, `Tuesday`, `Wednesday`, `Thursday`, `Friday`, `Saturday` | Yes | This value must be provided.
+maintenance-time-of-day-utc | The time (in UTC) when the maintenance window can begin on the AMLFS cluster. | Time value can only be in 24-hour format i.e., HH:MM | Yes | This value must be provided.
+location | Azure region in which the AMLFS cluster will be created. The region name should only have lower-case letters or numbers. | `eastus2`, `westus`, etc. | No | If empty, the driver will use the same region name as the current AKS cluster.
+resource-group-name | The name of the resource group in which to create the AMLFS cluster. This resource group must already exist. | Resource group names can only include alphanumeric characters, underscores, parentheses, hyphens, periods (except at the end), and Unicode characters that match the allowed characters. | No | If empty, the driver will use the AKS infrastructure resource group.
+vnet-resource-group | The name of the resource group containing the virtual network to be connected to the AMLFS cluster. This resource group must already exist. | Resource group names can only include alphanumeric characters, underscores, parentheses, hyphens, periods (except at the end), and Unicode characters that match the allowed characters. | No | If empty, the driver will use current AKS cluster's virtual network resource group
+vnet-name | The name of the virtual network to be connected to the AMLFS cluster. This virtual network must already exist. Setup any virtual network peerings beforehand. | The name must begin with a letter or number, end with a letter, number, or underscore, and may contain only letters, numbers, underscores, periods, or hyphens. | No | If empty, the driver will use current AKS cluster's virtual network
+subnet-name | The name of the subnet within the virtual network to be connected to the AMLFS cluster. This subnet must already exist. | The name must begin with a letter or number, end with a letter, number, or underscore, and may contain only letters, numbers, underscores, periods, or hyphens. | No | If empty, the driver will use current AKS cluster's subnet
+identities | User-assigned identities to assign to the AMLFS cluster. These identities must already exist. | This must be the resource identifier for the identity e.g., `"/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/myResourceGroup/providers/Microsoft.ManagedIdentity/userAssignedIdentities/myManagedIdentity"`. Multiple values may be provided as a comma-separated list. | No | None
+tags | Tags to apply to the AMLFS cluster resource. These tags do not affect AMLFS cluster functionality. | Tag format: `"key1=val1,key2=val2"`. The tag name has a limit of 512 characters and the tag value has a limit of 256 characters. Tag names can't contain these characters: `<, >, %, &, \, ?, /`. | No | None
+sub-dir | This is the subdirectory within the AMLFS cluster's root directory which is where each pod will actually be mounted within the AMLFS filesystem. This subdirectory does not need to exist beforehand. | This must be a valid Linux file path. It can also interpret metadata such as `"${pvc.metadata.name}"`, `"${pvc.metadata.namespace}"`, `"${pv.metadata.name}"`, `"${pod.metadata.name}"`, `"${pod.metadata.namespace}"`, `"${pod.metadata.uid}"`.| No | None, will default to mounting the root directory of the AMLFS cluster.
+
+## Static Provisioning (Bring your own AMLFS Cluster through AKS)
+
+Name | Meaning | Available Value | Mandatory | Default value
+--- | --- | --- | --- | ---
+mgs-ip-address | The IP address of the Lustre MGS, see AMLFS cluster details. | Must be a valid IP address i.e., `x.x.x.x` | Yes | This value must be provided.
+sub-dir | This is subdirectory within the AMLFS cluster's root directory which is where each pod will actually be mounted within the AMLFS filesystem. This subdirectory does not need to exist beforehand. | This must be a valid Linux file path. It can also interpret metadata such as `"${pvc.metadata.name}"`, `"${pvc.metadata.namespace}"`, `"${pv.metadata.name}"`, `"${pod.metadata.name}"`, `"${pod.metadata.namespace}"`, `"${pod.metadata.uid}"`.| No | None, will default to mounting the root directory of the AMLFS cluster.
diff --git a/docs/examples/pod_echo_date_dynprov.yaml b/docs/examples/pod_echo_date_dynprov.yaml
@@ -0,0 +1,22 @@
+---
+kind: Pod
+apiVersion: v1
+metadata:
+  name: lustre-echo-date-dynprov
+spec:
+  nodeSelector:
+    "kubernetes.io/os": linux
+  containers:
+    - image: mcr.microsoft.com/azurelinux/base/core:3.0
+      name: lustre-echo-date
+      command:
+        - "/bin/sh"
+        - "-c"
+        - while true; do echo $(date) >> /mnt/lustre/outfile; tail -1 /mnt/lustre/outfile; sleep 1; done
+      volumeMounts:
+        - name: lustre-dynprov
+          mountPath: "/mnt/lustre"
+  volumes:
+    - name: lustre-dynprov
+      persistentVolumeClaim:
+        claimName: pvc-lustre-dynprov
diff --git a/docs/examples/pv.yaml b/docs/examples/pv.yaml
@@ -14,8 +14,6 @@ spec:
   csi:
     driver: azurelustre.csi.azure.com
     volumeAttributes:
-      # The file system name of the existing Lustre
-      fs-name: ${EXISTING_LUSTRE_FS_NAME}
       # The IP address of the existing Lustre
       mgs-ip-address: ${EXISTING_LUSTRE_IP_ADDRESS}
     # Make sure this VolumeID is unique in the cluster

diff --git a/docs/examples/pv_subdir.yaml b/docs/examples/pv_subdir.yaml
@@ -14,8 +14,6 @@ spec:
   csi:
     driver: azurelustre.csi.azure.com
     volumeAttributes:
-      # The file system name of the existing Lustre
-      fs-name: ${EXISTING_LUSTRE_FS_NAME}
       # The IP address of the existing Lustre
       mgs-ip-address: ${EXISTING_LUSTRE_IP_ADDRESS}
       # The subdirectory to create and mount per pod (cannot use pvc / pv metadata in static volumes)

diff --git a/docs/examples/pvc_storageclass_dynprov.yaml b/docs/examples/pvc_storageclass_dynprov.yaml
@@ -0,0 +1,15 @@
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  # The name of the PVC
+  name: pvc-lustre-dynprov
+spec:
+  accessModes:
+    - ReadWriteMany
+  resources:
+    requests:
+      # The real storage capacity in the claim
+      storage: 48Ti
+  # This field must be the same as the storage class name in StorageClass
+  storageClassName: dynprov.azurelustre.csi.azure.com
diff --git a/docs/examples/storageclass_dynprov_lustre.yml b/docs/examples/storageclass_dynprov_lustre.yml
@@ -0,0 +1,66 @@
+---
+apiVersion: storage.k8s.io/v1
+kind: StorageClass
+metadata:
+  # The name of the StorageClass.
+  name: dynprov.azurelustre.csi.azure.com
+parameters:
+  # See the driver-parameters.md file for a full description of parameters.
+  #
+  # SKU name for the Azure Managed Lustre file system. The SKU determines the throughput of the AMLFS cluster.
+  # This value must be one of:
+  # "AMLFS-Durable-Premium-40"
+  # "AMLFS-Durable-Premium-125"
+  # "AMLFS-Durable-Premium-250"
+  # "AMLFS-Durable-Premium-500"
+  sku-name: {SKU_NAME}
+  #
+  # The availability zone where your resource will be created.
+  # For the best performance, locate your AMLFS cluster in the same region and availability zone that houses your AKS cluster and other compute clients.
+  # The zone must be a single value e.g., "1", "2", or "3".
+  zones: {ZONE}
+  #
+  # The day of the week for maintenance to be performed on the AMLFS cluster, "Sunday", "Monday", etc.
+  maintenance-day-of-week: {MAINTENANCE_DAY}
+  #
+  # The time (in UTC) when the maintenance window can begin on the AMLFS cluster. Time value can only be in 24-hour format i.e., HH:MM
+  maintenance-time-of-day-utc: {TIME_OF_DAY}
+  #
+  # Optional parameters:
+  #
+  # Azure region in which the AMLFS cluster will be created. The region name should only have lower-case letters or numbers.
+  # If empty, the driver will use the same region name as the current AKS cluster.
+  # location: {LOCATION}
+  #
+  # The name of the resource group in which to create the AMLFS cluster. This resource group must already exist.
+  # If empty, the driver will use the AKS infrastructure resource group.
+  # resource-group-name: {RESOURCE_GROUP_NAME}
+  #
+  # The name of the resource group containing the virtual network to be connected to the AMLFS cluster. This resource group must already exist.
+  # vnet-resource-group: {EXISTING_VNET_RG}
+  #
+  # The name of the virtual network to be connected to the AMLFS cluster. This virtual network must already exist. Setup any virtual network peerings beforehand.
+  # vnet-name: {EXISTING_VNET_NAME}
+  #
+  # The name of the subnet within the virtual network to be connected to the AMLFS cluster. This subnet must already exist.
+  # subnet-name: {EXISTING_SUBNET_NAME}
+  #
+  # User-assigned identities to assign to the AMLFS cluster. These identities must already exist. | This must be the resource identifier for the identity e.g., `"/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/myResourceGroup/providers/Microsoft.ManagedIdentity/userAssignedIdentities/myManagedIdentity"`
+  # identities: {IDENTITIES}
+  #
+  # Tags to apply to the AMLFS cluster resource. These tags do not affect AMLFS cluster functionality. Tag format: `"key1=val1,key2=val2"`.
+  # tags: {TAGS}
+  #
+  # The subdirectory within the AMLFS cluster's root directory which is where each pod will actually be mounted within the AMLFS filesystem.
+  # This subdirectory does not need to exist beforehand. This must be a valid Linux file path.
+  # It can also interpret metadata such as `"${pvc.metadata.name}"`, `"${pvc.metadata.namespace}"`, `"${pv.metadata.name}"`, `"${pod.metadata.name}"`, `"${pod.metadata.namespace}"`, `"${pod.metadata.uid}"`.
+  # sub-dir: {SUBDIRECTORY}
+  #
+provisioner: azurelustre.csi.azure.com
+# Azure Lustre Cluster is removed after PVC deletion if reclaimPolicy is the default value "Delete".
+# To keep the cluster after PVC deletion, set reclaimPolicy to "Retain".
+reclaimPolicy: Delete
+volumeBindingMode: Immediate
+mountOptions:
+  - noatime
+  - flock
diff --git a/docs/examples/storageclass_existing_lustre.yaml b/docs/examples/storageclass_existing_lustre.yaml
@@ -5,8 +5,6 @@ metadata:
   # The name of the StorageClass.
   name: sc.azurelustre.csi.azure.com
 parameters:
-  # The file system name of the existing Lustre, "lustrefs" in common case
-  fs-name: ${EXISTING_LUSTRE_FS_NAME}
   # The IP address of the existing Lustre
   mgs-ip-address: ${EXISTING_LUSTRE_IP_ADDRESS}
 provisioner: azurelustre.csi.azure.com

diff --git a/docs/examples/storageclass_existing_lustre_subdir.yaml b/docs/examples/storageclass_existing_lustre_subdir.yaml
@@ -5,8 +5,6 @@ metadata:
   # The name of the StorageClass.
   name: subdir.azurelustre.csi.azure.com
 parameters:
-  # The file system name of the existing Lustre, "lustrefs" in common case
-  fs-name: ${EXISTING_LUSTRE_FS_NAME}
   # The IP address of the existing Lustre
   mgs-ip-address: ${EXISTING_LUSTRE_IP_ADDRESS}
   # The subdirectory to create and mount per pod

diff --git a/docs/static-provisioning.md b/docs/static-provisioning.md
@@ -12,8 +12,7 @@ This document explains how to deploy your workload with an existing Azure Lustre
 [static provision storage class](./examples/storageclass_existing_lustre.yaml)
 and [PVC file](./examples/pvc_storageclass.yaml).
 
-* Edit the `EXISTING_LUSTRE_FS_NAME` and `EXISTING_LUSTRE_IP_ADDRESS` in
-storage class.
+* Edit the `EXISTING_LUSTRE_IP_ADDRESS` in the storage class.
 
 * Create storage class and `PVC`
 
@@ -29,8 +28,7 @@ kubectl create -f pvc_storageclass.yaml
 * Download [PV file](./examples/pv.yaml) and
 [PVC file](./examples/pvc_pv.yaml)
 
-* Edit the `EXISTING_LUSTRE_FS_NAME`, `EXISTING_LUSTRE_IP_ADDRESS` and
-`UNIQUE_IDENTIFIER_VOLUME_ID` in pv.
+* Edit the `EXISTING_LUSTRE_IP_ADDRESS` and `UNIQUE_IDENTIFIER_VOLUME_ID` in the pv.
 
 * Create PV and PVC.
 

diff --git a/go.mod b/go.mod
@@ -3,8 +3,11 @@ module sigs.k8s.io/azurelustre-csi-driver
 go 1.23.6
 
 require (
+	github.com/Azure/azure-sdk-for-go/sdk/azcore v1.17.0
 	github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.8.2
+	github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/network/armnetwork/v6 v6.2.0
 	github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/resources/armresources v1.2.0
+	github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storagecache/armstoragecache/v4 v4.0.0
 	github.com/Azure/go-autorest/autorest v0.11.30
 	github.com/container-storage-interface/spec v1.11.0
 	github.com/kubernetes-csi/csi-lib-utils v0.19.0
@@ -26,8 +29,18 @@ require (
 
 require (
 	github.com/Azure/azure-sdk-for-go v68.0.0+incompatible // indirect
-	github.com/Azure/azure-sdk-for-go/sdk/azcore v1.17.0 // indirect
 	github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 // indirect
+	github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/authorization/armauthorization/v2 v2.2.0 // indirect
+	github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v5 v5.7.0 // indirect
+	github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerregistry/armcontainerregistry v1.2.0 // indirect
+	github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v4 v4.8.0 // indirect
+	github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/keyvault/armkeyvault v1.4.0 // indirect
+	github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/msi/armmsi v1.2.0 // indirect
+	github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/network/armnetwork/v4 v4.3.0 // indirect
+	github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/privatedns/armprivatedns v1.2.0 // indirect
+	github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storage/armstorage v1.6.0 // indirect
+	github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/azsecrets v1.1.0 // indirect
+	github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/internal v1.0.0 // indirect
 	github.com/Azure/go-autorest v14.2.0+incompatible // indirect
 	github.com/Azure/go-autorest/autorest/adal v0.9.24 // indirect
 	github.com/Azure/go-autorest/autorest/date v0.3.0 // indirect
@@ -46,6 +59,7 @@ require (
 	github.com/fsnotify/fsnotify v1.7.0 // indirect
 	github.com/fxamacker/cbor/v2 v2.7.0 // indirect
 	github.com/go-logr/logr v1.4.2 // indirect
+	github.com/go-logr/stdr v1.2.2 // indirect
 	github.com/go-openapi/jsonpointer v0.21.0 // indirect
 	github.com/go-openapi/jsonreference v0.21.0 // indirect
 	github.com/go-openapi/swag v0.23.0 // indirect
@@ -77,32 +91,46 @@ require (
 	github.com/opencontainers/runtime-spec v1.2.0 // indirect
 	github.com/opencontainers/selinux v1.11.1 // indirect
 	github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect
+	github.com/pkg/errors v0.9.1 // indirect
 	github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
 	github.com/prometheus/client_golang v1.20.5 // indirect
 	github.com/prometheus/client_model v0.6.1 // indirect
 	github.com/prometheus/common v0.60.0 // indirect
 	github.com/prometheus/procfs v0.15.1 // indirect
-	github.com/rogpeppe/go-internal v1.13.1 // indirect
 	github.com/sirupsen/logrus v1.9.3 // indirect
 	github.com/spf13/pflag v1.0.5 // indirect
 	github.com/x448/float16 v0.8.4 // indirect
+	go.opentelemetry.io/auto/sdk v1.1.0 // indirect
+	go.opentelemetry.io/otel v1.34.0 // indirect
+	go.opentelemetry.io/otel/exporters/prometheus v0.50.0 // indirect
+	go.opentelemetry.io/otel/metric v1.34.0 // indirect
+	go.opentelemetry.io/otel/sdk v1.34.0 // indirect
+	go.opentelemetry.io/otel/sdk/metric v1.34.0 // indirect
+	go.opentelemetry.io/otel/trace v1.34.0 // indirect
+	go.uber.org/mock v0.4.0
 	golang.org/x/crypto v0.36.0 // indirect
 	golang.org/x/oauth2 v0.25.0 // indirect
+	golang.org/x/sync v0.12.0 // indirect
 	golang.org/x/sys v0.31.0 // indirect
 	golang.org/x/term v0.30.0 // indirect
 	golang.org/x/text v0.23.0 // indirect
 	golang.org/x/time v0.7.0 // indirect
 	golang.org/x/tools v0.26.0 // indirect
 	google.golang.org/genproto/googleapis/rpc v0.0.0-20250303144028-a0af3efb3deb // indirect
+	gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect
 	gopkg.in/inf.v0 v0.9.1 // indirect
 	gopkg.in/yaml.v2 v2.4.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 	k8s.io/api v0.31.6 // indirect
 	k8s.io/apiextensions-apiserver v0.31.1 // indirect
 	k8s.io/apiserver v0.31.6 // indirect
 	k8s.io/client-go v1.5.2 // indirect
+	k8s.io/cloud-provider v0.31.1 // indirect
 	k8s.io/component-base v0.31.6 // indirect
+	k8s.io/component-helpers v0.31.6 // indirect
 	k8s.io/kube-openapi v0.0.0-20241009091222-67ed5848f094 // indirect
+	sigs.k8s.io/cloud-provider-azure/pkg/azclient v0.0.55 // indirect
+	sigs.k8s.io/cloud-provider-azure/pkg/azclient/configloader v0.1.5
 	sigs.k8s.io/json v0.0.0-20241014173422-cfa47c3a1cc8 // indirect
 	sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect
 )