设置 Azure 云提供商
使用 Azure
云提供商时,你可以使用以下功能:
负载均衡器:在特定网络安全组中启动 Azure 负载均衡器。
持久卷:支持将 Azure Blob 磁盘和 Azure 托管磁盘与标准和高级 Storage Account 一起使用。
网络存储:通过 CIFS 挂载支持 Azure 文件。
Azure 订阅不支持以下账号类型:
- 单租户账号(即没有订阅的账号)。
- 多订阅账号。
RKE 和 RKE2 的先决条件
要为 RKE 和 RKE2 设置 Azure 云提供商,你需要配置以下凭证:
1. 设置 Azure 租户 ID
访问 Azure 门户,登录并转到 Azure Active Directory,然后选择 Properties。你的 Directory ID 是你的 Tenant ID (tenantID)。
如果要使用 Azure CLI,你可以运行 az account show
命令来获取信息。
2. 设置 Azure 客户端 ID 和 Azure 客户端密码
访问 Azure 门户,登录并按照以下步骤创建 App Registration 和对应的 Azure Client ID (aadClientId) 以及 Azure Client Secret (aadClientSecret)。
- 选择 Azure Active Directory。
- 选择 App registrations。
- 选择 New application registration。
- 选择 Name,选择
Web app/API
作为 Application Type,并选择任意 Sign-on URL。 - 选择 Create。
在 App registrations 视图中,你应该会看到你创建的应用注册。APPLICATION ID 列中显示的值是需要用作 Azure Client ID 的值。
下一步是生成 Azure Client Secret:
- 打开你创建的应用注册。
- 在 Settings 视图中,打开 Keys。
- 输入 Key description,选择过期时间,然后选择 Save。
- Value 列中显示的生成值是需要用作 Azure Client Secret 的值。该值只会显示一次。
3. 配置应用注册权限
最后,为你的应用注册分配适当的权限:
- 前往 More services,搜索 Subscriptions 并打开它。
- 打开 Access control (IAM)。
- 选择 Add。
- 在 Role 中选择
Contributor
。 - 在 Select 中选择你创建的应用注册的名称。
- 选择 Save。
4. 设置 Azure 网络安全组名称
要使 Azure 负载均衡器正常工作,你需要自定义一个 Azure 网络安全组 (securityGroupName)。
如果你使用 Rancher Machine Azure 驱动来配置主机,则需要手动编辑主机,从而将主机分配给此网络安全组。
你需要在配置期间将自定义主机分配给此网络安全组。
只有需要成为负载均衡器后端的主机才需要分配到该组。
Rancher 中的 RKE2 集群设置
在集群配置中的云提供商下拉列表中选择 Azure。
- 配置云提供商。请注意,Rancher 会自动创建新的网络安全组、资源组、可用性集、子网和虚拟网络。如果你已经创建了其中的一部分或全部,则需要在创建集群之前指定它们。
- 你可以单击显示高级选项以查看更多自动生成的名称,并在需要的时候更新它们。你的云提供商配置必须与主机池中的字段匹配。如果你有多个池,它们必须使用相同的资源组、可用性集、子网、虚拟网络和网络安全组。
- 下面提供了一个示例。你可以根据需要对其进行修改。
云提供商配置示例
{
"cloud":"AzurePublicCloud",
"tenantId": "YOUR TENANTID HERE",
"aadClientId": "YOUR AADCLIENTID HERE",
"aadClientSecret": "YOUR AADCLIENTSECRET HERE",
"subscriptionId": "YOUR SUBSCRIPTIONID HERE",
"resourceGroup": "docker-machine",
"location": "westus",
"subnetName": "docker-machine",
"securityGroupName": "rancher-managed-KA4jV9V2",
"securityGroupResourceGroup": "docker-machine",
"vnetName": "docker-machine-vnet",
"vnetResourceGroup": "docker-machine",
"primaryAvailabilitySetName": "docker-machine",
"routeTableResourceGroup": "docker-machine",
"cloudProviderBackoff": false,
"useManagedIdentityExtension": false,
"useInstanceMetadata": true
}
在集群配置 > 高级选项、中,单击补充的 Controller Manager 参数下的添加,并添加
--configure-cloud-routes=false
标志。单击创建按钮来提交表单并创建集群。
Cloud Provider Configuration
Rancher automatically creates a new Network Security Group, Resource Group, Availability Set, Subnet, and Virtual Network. If you already have some or all of these created, you will need to specify them before creating the cluster. You can check RKE1 Node Templates or RKE2 Machine Pools to view or edit these automatically generated names.
Refer to the full list of configuration options in the upstream docs.
useInstanceMetadata
must be set totrue
for the cloud provider to correctly configureproviderID
.excludeMasterFromStandardLB
must be set tofalse
if you need to add nodes labelednode-role.kubernetes.io/master
to the backend of the Azure Load Balancer (ALB).loadBalancerSku
can be set tobasic
orstandard
. Basic SKU will be deprecated in September 2025. Refer to the Azure upstream docs for more information.
Azure supports reading the cloud config from Kubernetes secrets. The secret is a serialized version of the azure.json file. When the secret is changed, the cloud controller manager reconstructs itself without restarting the pod. It is recommended for the Helm chart to read the Cloud Provider Config from the secret.
Note that the chart reads the Cloud Provider Config from a given secret name in the kube-system
namespace. Since Azure reads Kubernetes secrets, RBAC also needs to be configured. An example secret for the Cloud Provider Config is shown below. Modify it as needed and create the secret.
# azure-cloud-config.yaml
apiVersion: v1
kind: Secret
metadata:
name: azure-cloud-config
namespace: kube-system
type: Opaque
stringData:
cloud-config: |-
{
"cloud": "AzurePublicCloud",
"tenantId": "<tenant-id>",
"subscriptionId": "<subscription-id>",
"aadClientId": "<client-id>",
"aadClientSecret": "<tenant-id>",
"resourceGroup": "docker-machine",
"location": "westus",
"subnetName": "docker-machine",
"securityGroupName": "rancher-managed-kqmtsjgJ",
"securityGroupResourceGroup": "docker-machine",
"vnetName": "docker-machine-vnet",
"vnetResourceGroup": "docker-machine",
"primaryAvailabilitySetName": "docker-machine",
"routeTableResourceGroup": "docker-machine",
"cloudProviderBackoff": false,
"useManagedIdentityExtension": false,
"useInstanceMetadata": true,
"loadBalancerSku": "standard",
"excludeMasterFromStandardLB": false,
}
Using the Out-of-tree Azure Cloud Provider
- RKE2
- RKE1
Select External from the Cloud Provider drop-down in the Cluster Configuration section.
Under Cluster Configuration > Advanced, click Add under Additional Controller Manager Args and add this flag:
--configure-cloud-routes=false
.Prepare the Cloud Provider Configuration to set it in the next step. Note that Rancher automatically creates a new Network Security Group, Resource Group, Availability Set, Subnet, and Virtual Network. If you already have some or all of these created, you must specify them before creating the cluster.
Click Show Advanced to view or edit these automatically generated names. Your Cloud Provider Configuration must match the fields in the Machine Pools section. If you have multiple pools, they must all use the same Resource Group, Availability Set, Subnet, Virtual Network, and Network Security Group.
Under Cluster Configuration > Add-on Config, add the cloud controller manager manifest shown below into Additional Manifest. Note that this chart reads the Cloud Provider Config from the secret in the
kube-system
namespace. An example secret for the Cloud Provider Config is shown below; modify it as needed. Refer to the full list of configuration options in the upstream docs.Alternatively, you can also install the cloud controller manager using the Helm CLI.
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
name: azure-cloud-controller-manager
namespace: kube-system
spec:
chart: cloud-provider-azure
repo: https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/helm/repo
targetNamespace: kube-system
bootstrap: true
valuesContent: |-
infra:
clusterName: <cluster-name>
cloudControllerManager:
cloudConfigSecretName: azure-cloud-config
cloudConfig: null
clusterCIDR: null
enableDynamicReloading: 'true'
nodeSelector:
node-role.kubernetes.io/control-plane: 'true'
allocateNodeCidrs: 'false'
hostNetworking: true
caCertDir: /etc/ssl
configureCloudRoutes: 'false'
enabled: true
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
- effect: NoSchedule
key: node-role.kubernetes.io/control-plane
value: 'true'
- effect: NoSchedule
key: node.cloudprovider.kubernetes.io/uninitialized
value: 'true'
---
apiVersion: v1
kind: Secret
metadata:
name: azure-cloud-config
namespace: kube-system
type: Opaque
stringData:
cloud-config: |-
{
"cloud": "AzurePublicCloud",
"tenantId": "<tenant-id>",
"subscriptionId": "<subscription-id>",
"aadClientId": "<client-id>",
"aadClientSecret": "<tenant-id>",
"resourceGroup": "docker-machine",
"location": "westus",
"subnetName": "docker-machine",
"securityGroupName": "rancher-managed-kqmtsjgJ",
"securityGroupResourceGroup": "docker-machine",
"vnetName": "docker-machine-vnet",
"vnetResourceGroup": "docker-machine",
"primaryAvailabilitySetName": "docker-machine",
"routeTableResourceGroup": "docker-machine",
"cloudProviderBackoff": false,
"useManagedIdentityExtension": false,
"useInstanceMetadata": true,
"loadBalancerSku": "standard",
"excludeMasterFromStandardLB": false,
}Click Create to submit the form and create the cluster.
Choose External from the Cloud Provider drop-down in the Cluster Options section. This sets
--cloud-provider=external
for Kubernetes components.Install the
cloud-provider-azure
chart after the cluster finishes provisioning. Note that the cluster is not successfully provisioned and nodes are still in anuninitialized
state until you deploy the cloud controller manager. This can be done manually using CLI, or via Helm charts in UI.
Refer to the official Azure upstream documentation for more details on deploying the Cloud Controller Manager.
Helm Chart Installation from CLI
Official upstream docs for Helm chart installation can be found on Github.
- Create a
azure-cloud-config
secret with the required cloud provider config.
kubectl apply -f azure-cloud-config.yaml
- Add the Helm repository:
helm repo add azure-cloud-controller-manager https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/helm/repo
helm repo update
- Create a
values.yaml
file with the following contents to override the defaultvalues.yaml
:
- RKE2
- RKE
# values.yaml
infra:
clusterName: <cluster-name>
cloudControllerManager:
cloudConfigSecretName: azure-cloud-config
cloudConfig: null
clusterCIDR: null
enableDynamicReloading: 'true'
configureCloudRoutes: 'false'
allocateNodeCidrs: 'false'
caCertDir: /etc/ssl
enabled: true
replicas: 1
hostNetworking: true
nodeSelector:
node-role.kubernetes.io/control-plane: 'true'
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
- effect: NoSchedule
key: node-role.kubernetes.io/control-plane
value: 'true'
- effect: NoSchedule
key: node.cloudprovider.kubernetes.io/uninitialized
value: 'true'
# values.yaml
cloudControllerManager:
cloudConfigSecretName: azure-cloud-config
cloudConfig: null
clusterCIDR: null
enableDynamicReloading: 'true'
configureCloudRoutes: 'false'
allocateNodeCidrs: 'false'
caCertDir: /etc/ssl
enabled: true
replicas: 1
hostNetworking: true
nodeSelector:
node-role.kubernetes.io/controlplane: 'true'
node-role.kubernetes.io/control-plane: null
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/controlplane
value: 'true'
- effect: NoSchedule
key: node.cloudprovider.kubernetes.io/uninitialized
value: 'true'
infra:
clusterName: <cluster-name>
- Install the Helm chart:
helm upgrade --install cloud-provider-azure azure-cloud-controller-manager/cloud-provider-azure -n kube-system --values values.yaml
Verify that the Helm chart installed successfully:
helm status cloud-provider-azure -n kube-system
- (Optional) Verify that the cloud controller manager update succeeded:
kubectl rollout status deployment -n kube-system cloud-controller-manager
kubectl rollout status daemonset -n kube-system cloud-node-manager
- The cloud provider is responsible for setting the ProviderID of the node. Check if all nodes are initialized with the ProviderID:
kubectl describe nodes | grep "ProviderID"
Helm Chart Installation from UI
Click ☰, then select the name of the cluster from the left navigation.
Select Apps > Repositories.
Click the Create button.
Enter
https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/helm/repo
in the Index URL field.Select Apps > Charts from the left navigation and install cloud-provider-azure chart.
Select the namespace,
kube-system
, and enable Customize Helm options before install.Replace
cloudConfig: /etc/kubernetes/azure.json
to read from the Cloud Config Secret and enable dynamic reloading:
cloudConfigSecretName: azure-cloud-config
enableDynamicReloading: 'true'
- Update the following fields as required:
allocateNodeCidrs: 'false'
configureCloudRoutes: 'false'
clusterCIDR: null
- RKE2
- RKE
- Rancher-provisioned RKE2 nodes have the selector
node-role.kubernetes.io/control-plane
set totrue
. Update the nodeSelector:
nodeSelector:
node-role.kubernetes.io/control-plane: 'true'
- Rancher-provisioned RKE nodes are tainted
node-role.kubernetes.io/controlplane
. Update tolerations and the nodeSelector:
tolerations:
- effect: NoSchedule
key: node.cloudprovider.kubernetes.io/uninitialized
value: 'true'
- effect: NoSchedule
value: 'true'
key: node-role.kubernetes.io/controlplane
nodeSelector:
node-role.kubernetes.io/controlplane: 'true'
- Install the chart and confirm that the cloud controller and cloud node manager deployed successfully:
kubectl rollout status deployment -n kube-system cloud-controller-manager
kubectl rollout status daemonset -n kube-system cloud-node-manager
- The cloud provider is responsible for setting the ProviderID of the node. Check if all nodes are initialized with the ProviderID:
kubectl describe nodes | grep "ProviderID"
Installing CSI Drivers
Install Azure Disk CSI driver or Azure File CSI Driver to access Azure Disk or Azure File volumes respectively.
The steps to install the Azure Disk CSI driver are shown below. You can install the Azure File CSI Driver in a similar manner by following the helm installation documentation.
::: note Important:
Clusters must be provisioned using Managed Disk
to use Azure Disk. You can configure this when creating RKE1 Node Templates or *RKE2 Machine Pools.
:::
Official upstream docs for Helm chart installation can be found on Github.
- Add and update the helm repository:
helm repo add azuredisk-csi-driver https://raw.githubusercontent.com/kubernetes-sigs/azuredisk-csi-driver/master/charts
helm repo update azuredisk-csi-driver
- Install the chart as shown below, updating the --version argument as needed. Refer to the full list of latest chart configurations in the upstream docs.
helm install azuredisk-csi-driver azuredisk-csi-driver/azuredisk-csi-driver --namespace kube-system --version v1.30.1 --set controller.cloudConfigSecretName=azure-cloud-config --set controller.cloudConfigSecretNamespace=kube-system --set controller.runOnControlPlane=true
- (Optional) Verify that the azuredisk-csi-driver installation succeeded:
kubectl --namespace=kube-system get pods --selector="app.kubernetes.io/name=azuredisk-csi-driver" --watch
- Provision an example Storage Class:
cat <<EOF | kubectl create -f -
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: standard
provisioner: kubernetes.io/azure-disk
parameters:
storageaccounttype: Standard_LRS
kind: Managed
EOF
Verify that the storage class has been provisioned:
kubectl get storageclasses
- Create a PersistentVolumeClaim:
cat <<EOF | kubectl create -f -
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: azure-disk-pvc
spec:
storageClassName: standard
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
EOF
Verify that the PersistentVolumeClaim and PersistentVolume have been created:
kubectl get persistentvolumeclaim
kubectl get persistentvolume
- Attach the new Azure Disk:
You can now mount the Kubernetes PersistentVolume into a Kubernetes Pod. The disk can be consumed by any Kubernetes object type, including a Deployment, DaemonSet, or StatefulSet. However, the following example simply mounts the PersistentVolume into a standalone Pod.
cat <<EOF | kubectl create -f -
kind: Pod
apiVersion: v1
metadata:
name: mypod-dynamic-azuredisk
spec:
containers:
- name: mypod
image: nginx
ports:
- containerPort: 80
name: "http-server"
volumeMounts:
- mountPath: "/usr/share/nginx/html"
name: storage
volumes:
- name: storage
persistentVolumeClaim:
claimName: azure-disk-pvc
EOF