azure-aks

Azure Kubernetes Service

使用 AKS 在 Azure 上部署和管理可用于生产环境的 Kubernetes 集群。涵盖集群创建、节点池管理、网络、入口控制器、监控、安全性和基于 Terraform 的预配。

使用场景

• 你需要托管式 Kubernetes，无需维护控制平面基础设施。
• 你的工作负载需要具备自动扩缩能力的容器编排。
• 你需要与 Azure AD、Key Vault 和 Container Registry 紧密集成。
• 你正在运行需要服务网格、入口或网络策略的微服务。
• 你需要 GPU 或 Spot 节点池来运行专用型或成本优化型工作负载。

先决条件

# Install Azure CLI and kubectl
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
az aks install-cli

# Login and set subscription
az login
az account set --subscription "my-subscription-id"

# Register required providers
az provider register --namespace Microsoft.ContainerService
az provider register --namespace Microsoft.OperationsManagement

# Verify kubectl
kubectl version --client

集群创建

基本生产集群

# Create resource group
az group create --name myapp-rg --location eastus

# Create AKS cluster with best-practice defaults
az aks create \
  --resource-group myapp-rg \
  --name myapp-aks \
  --node-count 3 \
  --node-vm-size Standard_D4s_v5 \
  --enable-managed-identity \
  --enable-cluster-autoscaler \
  --min-count 2 \
  --max-count 10 \
  --network-plugin azure \
  --network-policy calico \
  --service-cidr 10.1.0.0/16 \
  --dns-service-ip 10.1.0.10 \
  --vnet-subnet-id "/subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.Network/virtualNetworks/{vnet}/subnets/{subnet}" \
  --enable-aad \
  --aad-admin-group-object-ids "{aad-group-id}" \
  --enable-azure-rbac \
  --zones 1 2 3 \
  --generate-ssh-keys \
  --tags environment=prod team=platform

# Get cluster credentials
az aks get-credentials --resource-group myapp-rg --name myapp-aks

# Verify cluster access
kubectl get nodes -o wide
kubectl cluster-info

专用集群

az aks create \
  --resource-group myapp-rg \
  --name myapp-private-aks \
  --node-count 3 \
  --node-vm-size Standard_D4s_v5 \
  --enable-managed-identity \
  --enable-private-cluster \
  --private-dns-zone system \
  --network-plugin azure \
  --vnet-subnet-id "/subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.Network/virtualNetworks/{vnet}/subnets/{subnet}" \
  --generate-ssh-keys

节点池管理

# Add a user node pool for application workloads
az aks nodepool add \
  --resource-group myapp-rg \
  --cluster-name myapp-aks \
  --name apppool \
  --node-count 3 \
  --node-vm-size Standard_D8s_v5 \
  --mode User \
  --enable-cluster-autoscaler \
  --min-count 2 \
  --max-count 15 \
  --zones 1 2 3 \
  --labels workload=app tier=frontend \
  --node-taints dedicated=app:NoSchedule \
  --max-pods 50

# Add GPU node pool for ML workloads
az aks nodepool add \
  --resource-group myapp-rg \
  --cluster-name myapp-aks \
  --name gpupool \
  --node-count 1 \
  --node-vm-size Standard_NC6s_v3 \
  --mode User \
  --enable-cluster-autoscaler \
  --min-count 0 \
  --max-count 4 \
  --node-taints sku=gpu:NoSchedule \
  --labels workload=ml

# Add spot instance pool for batch workloads
az aks nodepool add \
  --resource-group myapp-rg \
  --cluster-name myapp-aks \
  --name spotpool \
  --node-count 2 \
  --node-vm-size Standard_D4s_v5 \
  --priority Spot \
  --eviction-policy Delete \
  --spot-max-price -1 \
  --enable-cluster-autoscaler \
  --min-count 0 \
  --max-count 20 \
  --labels workload=batch

# Scale a node pool manually
az aks nodepool scale \
  --resource-group myapp-rg \
  --cluster-name myapp-aks \
  --name apppool \
  --node-count 5

# Upgrade a node pool
az aks nodepool upgrade \
  --resource-group myapp-rg \
  --cluster-name myapp-aks \
  --name apppool \
  --kubernetes-version 1.28.3

# List node pools
az aks nodepool list \
  --resource-group myapp-rg \
  --cluster-name myapp-aks \
  --output table

入口控制器设置

# Install NGINX ingress controller via Helm
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx \
  --create-namespace \
  --set controller.replicaCount=2 \
  --set controller.nodeSelector."kubernetes\.io/os"=linux \
  --set controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-health-probe-request-path"=/healthz \
  --set controller.service.externalTrafficPolicy=Local

# Verify the ingress controller and get external IP
kubectl get svc -n ingress-nginx

Ingress 资源示例

# ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myapp-ingress
  namespace: myapp
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: "50m"
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - myapp.example.com
      secretName: myapp-tls
  rules:
    - host: myapp.example.com
      http:
        paths:
          - path: /api
            pathType: Prefix
            backend:
              service:
                name: api-service
                port:
                  number: 80
          - path: /
            pathType: Prefix
            backend:
              service:
                name: frontend-service
                port:
                  number: 80

监控和日志记录

# Enable Container Insights
az aks enable-addons \
  --resource-group myapp-rg \
  --name myapp-aks \
  --addons monitoring \
  --workspace-resource-id "/subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.OperationalInsights/workspaces/{workspace}"

# Enable Azure Policy add-on
az aks enable-addons \
  --resource-group myapp-rg \
  --name myapp-aks \
  --addons azure-policy

# Enable Key Vault secrets provider
az aks enable-addons \
  --resource-group myapp-rg \
  --name myapp-aks \
  --addons azure-keyvault-secrets-provider

# View cluster diagnostics
az aks show \
  --resource-group myapp-rg \
  --name myapp-aks \
  --query "addonProfiles" \
  --output table

# Install Prometheus + Grafana via Helm
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install kube-prometheus prometheus-community/kube-prometheus-stack \
  --namespace monitoring \
  --create-namespace \
  --set grafana.adminPassword='SecureGrafanaP@ss'

ACR 集成

# Create Azure Container Registry
az acr create \
  --resource-group myapp-rg \
  --name myappacr \
  --sku Standard

# Attach ACR to AKS (grants AcrPull role)
az aks update \
  --resource-group myapp-rg \
  --name myapp-aks \
  --attach-acr myappacr

# Build and push image
az acr build \
  --registry myappacr \
  --image myapp:v1.0 \
  --file Dockerfile .

# Verify pull access
kubectl run test --image=myappacr.azurecr.io/myapp:v1.0 --rm -it --restart=Never -- echo "ACR pull works"

Terraform 配置

resource "azurerm_kubernetes_cluster" "aks" {
  name                = "myapp-aks"
  location            = azurerm_resource_group.main.location
  resource_group_name = azurerm_resource_group.main.name
  dns_prefix          = "myapp"
  kubernetes_version  = "1.28"

  default_node_pool {
    name                = "system"
    vm_size             = "Standard_D4s_v5"
    enable_auto_scaling = true
    min_count           = 2
    max_count           = 5
    zones               = [1, 2, 3]
    vnet_subnet_id      = azurerm_subnet.aks.id

    node_labels = {
      role = "system"
    }
  }

  identity {
    type = "SystemAssigned"
  }

  network_profile {
    network_plugin    = "azure"
    network_policy    = "calico"
    service_cidr      = "10.1.0.0/16"
    dns_service_ip    = "10.1.0.10"
    load_balancer_sku = "standard"
  }

  azure_active_directory_role_based_access_control {
    managed                = true
    azure_rbac_enabled     = true
    admin_group_object_ids = [var.aks_admin_group_id]
  }

  oms_agent {
    log_analytics_workspace_id = azurerm_log_analytics_workspace.main.id
  }

  key_vault_secrets_provider {
    secret_rotation_enabled = true
  }

  tags = var.tags
}

resource "azurerm_kubernetes_cluster_node_pool" "app" {
  name                  = "app"
  kubernetes_cluster_id = azurerm_kubernetes_cluster.aks.id
  vm_size               = "Standard_D8s_v5"
  enable_auto_scaling   = true
  min_count             = 2
  max_count             = 15
  zones                 = [1, 2, 3]
  vnet_subnet_id        = azurerm_subnet.aks.id

  node_labels = {
    workload = "app"
  }

  node_taints = [
    "dedicated=app:NoSchedule"
  ]

  tags = var.tags
}

集群升级

# Check available Kubernetes versions
az aks get-upgrades \
  --resource-group myapp-rg \
  --name myapp-aks \
  --output table

# Upgrade control plane first
az aks upgrade \
  --resource-group myapp-rg \
  --name myapp-aks \
  --kubernetes-version 1.28.3 \
  --control-plane-only

# Then upgrade each node pool
az aks nodepool upgrade \
  --resource-group myapp-rg \
  --cluster-name myapp-aks \
  --name apppool \
  --kubernetes-version 1.28.3

# Check upgrade status
az aks show \
  --resource-group myapp-rg \
  --name myapp-aks \
  --query "provisioningState"

故障排除

症状	原因	修复方法
节点处于 NotReady 状态	VM 资源耗尽或网络问题	运行 kubectl describe node <name> 并检查事件；如有需要则扩容
Pod 卡在 Pending 状态	没有可用节点或资源请求过高	使用 az aks show 检查自动扩缩器状态；调整资源请求
ImagePullBackOff 错误	未附加 ACR 或镜像标签错误	使用 az aks check-acr --name myapp-aks --acr myappacr.azurecr.io 验证
Ingress 返回 404	Ingress 规范中的服务或路径不匹配	验证 kubectl get ingress 和服务端点
无法访问专用集群	未配置 VPN 或专用终结点	使用 az aks command invoke 或配置专用 DNS 解析
集群自动扩缩器未扩容	未设置 Pod 资源请求	为所有 Pod 定义 CPU／内存请求，以便调度器计算需求
Azure Policy 违规导致 Pod 无法运行	应用了限制性策略	检查 kubectl get constrainttemplate 并调整策略分配
持久卷未绑定	StorageClass 不匹配或区域问题	验证 kubectl get pvc，并确保 StorageClass 与节点池区域一致

相关技能

• terraform-azure -- 使用 Terraform 预配 AKS 集群，实现可重复的基础设施部署。
• azure-networking -- 配置 Azure CNI 所需的 VNet 和子网。
• arm-templates -- 将基于 Bicep 的 AKS 部署作为 Terraform 的替代方案。
• azure-vms -- 了解用于选择节点池的 VM 大小。