Cluster Autoscaling

Autoscaling is how a Kubernetes cluster matches capacity to demand without a human editing replica counts at 3 a.m. It operates on two independent axes: scaling pods (running more or fewer copies of your workload) and scaling nodes (adding or removing machines so those pods have somewhere to run). Get this right and you absorb traffic spikes gracefully while paying only for what you use; get it wrong and you either drop requests under load or burn money on idle capacity. This lesson covers the three controllers that make it work — the Horizontal Pod Autoscaler, the Vertical Pod Autoscaler, and the Cluster Autoscaler — plus the resource governance that keeps them honest.

Learning Objectives

Configure a HorizontalPodAutoscaler with CPU, memory, and custom metrics
Set correct resource requests and limits so autoscaling has a baseline
Right-size workloads with the Vertical Pod Autoscaler
Scale node pools automatically with the Cluster Autoscaler
Enforce fair usage with ResourceQuota and LimitRange

Requests and Limits: the Foundation

Autoscaling is meaningless without resource requests. The scheduler uses requests to place pods, and the HPA computes utilization as current usage ÷ requested amount. A pod with no CPU request has undefined utilization, so the HPA cannot scale it.

yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 2
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
    spec:
      containers:
        - name: api
          image: registry.example.com/api:1.4.0
          resources:
            requests:
              cpu: "250m"      # baseline the scheduler reserves
              memory: "256Mi"
            limits:
              cpu: "1000m"     # hard ceiling before CPU throttling
              memory: "512Mi"  # exceed this and the container is OOMKilled

Pro Tip

Set CPU limits generously (or omit them) but always set memory limits. CPU over-limit only throttles the container; memory over-limit gets it OOMKilled. Requests should reflect the typical steady-state usage, not the peak.

Horizontal Pod Autoscaler (HPA)

The HPA adds and removes pod replicas based on observed metrics. It requires the metrics-server to be installed for CPU and memory, and a custom-metrics adapter (such as Prometheus Adapter) for anything else. Always use the autoscaling/v2 API — v1 only supports a single CPU target and is effectively deprecated.

yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300  # wait 5 min of low load before shrinking
      policies:
        - type: Percent
          value: 50
          periodSeconds: 60
    scaleUp:
      stabilizationWindowSeconds: 0    # react to spikes immediately
      policies:
        - type: Percent
          value: 100
          periodSeconds: 30

When multiple metrics are listed, the HPA calculates a desired replica count for each and picks the highest — so the busiest dimension wins. The behavior block (available since v2) is what stops "flapping": a long scaleDown stabilization window keeps replicas up through short traffic dips, while an aggressive scaleUp policy lets you double capacity in seconds.

Scaling on custom and external metrics

CPU is a poor proxy for many workloads. A queue consumer should scale on queue depth; an HTTP service on requests per second. With a metrics adapter installed you can target those directly.

yaml
metrics:
  - type: Pods
    pods:
      metric:
        name: http_requests_per_second
      target:
        type: AverageValue
        averageValue: "500"   # scale so each pod handles ~500 rps

Note

For event-driven scaling — Kafka, RabbitMQ, SQS, cron, or scale-to-zero — reach for KEDA. It ships 60+ scalers and generates a managed HPA under the hood, so you keep the same controller but get far richer triggers.

Vertical Pod Autoscaler (VPA)

Where the HPA changes how many pods run, the VPA changes how big each pod is by adjusting CPU and memory requests. It is ideal for workloads that cannot scale horizontally — a single-writer database, a leader-elected controller — or for discovering correct request values in the first place.

yaml
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: worker-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: worker
  updatePolicy:
    updateMode: "Auto"   # Off | Initial | Recreate | Auto
  resourcePolicy:
    containerPolicies:
      - containerName: "*"
        minAllowed:
          cpu: "100m"
          memory: "128Mi"
        maxAllowed:
          cpu: "2"
          memory: "2Gi"
        controlledResources: ["cpu", "memory"]

The updateMode values matter:

Off — only produces recommendations; apply them yourself. Safest starting point.
Initial — sets requests when a pod is created, never touches running pods.
Recreate / Auto — evicts and recreates pods to apply new requests live.

Warning

Do not point an HPA and a VPA at the same CPU or memory metric — they fight, since one changes replica count while the other changes the request the HPA measures against. It is safe to combine them only when the HPA scales on a custom metric (e.g. RPS) and the VPA manages CPU/memory. Also note that in Auto mode the VPA evicts pods, so protect availability with a PodDisruptionBudget.

Cluster Autoscaler: scaling the nodes

Pods that cannot be scheduled because every node is full get stuck in Pending. The Cluster Autoscaler (CA) watches for exactly this and provisions new nodes from your cloud provider's node groups (an EKS managed node group, a GKE node pool, an AKS agent pool). When nodes sit underutilized and their pods can fit elsewhere, it drains and removes them.

yaml
# Node-group discovery via tags on AWS EKS
command:
  - ./cluster-autoscaler
  - --cloud-provider=aws
  - --namespace=kube-system
  - --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/my-cluster
  - --balance-similar-node-groups
  - --expander=least-waste          # pick the group that leaves the least idle capacity
  - --scale-down-utilization-threshold=0.5
  - --scale-down-unneeded-time=10m  # node must be idle this long before removal

The CA reasons purely about resource requests, not live usage — a node whose pods request 100% of CPU but idle at 5% is considered full. This is another reason accurate requests matter. To keep the autoscaler from evicting a critical pod during scale-down, annotate it:

bash
kubectl annotate pod payments-processor \
  cluster-autoscaler.kubernetes.io/safe-to-evict=false

Works against fixed node groups you define up front. Mature, cloud-agnostic, and predictable — you choose the instance types, it changes the count.

Governing Capacity: ResourceQuota and LimitRange

Autoscaling amplifies mistakes — a single misconfigured deployment can scale to hundreds of pods and exhaust a shared cluster. Namespace-level guardrails contain the blast radius.

yaml
apiVersion: v1
kind: ResourceQuota
metadata:
  name: team-quota
  namespace: team-a
spec:
  hard:
    requests.cpu: "20"
    requests.memory: 40Gi
    limits.cpu: "40"
    limits.memory: 80Gi
    pods: "100"
---
apiVersion: v1
kind: LimitRange
metadata:
  name: default-limits
  namespace: team-a
spec:
  limits:
    - type: Container
      default:            # applied as limits if none specified
        cpu: "500m"
        memory: 256Mi
      defaultRequest:     # applied as requests if none specified
        cpu: "100m"
        memory: 128Mi

Warning

Once a ResourceQuota sets requests.cpu or limits.memory, every pod in that namespace must declare the corresponding value or the API server rejects it. Pair the quota with a LimitRange so bare pods still get sensible defaults instead of failing admission.

Common Pitfalls

Best Practices

Production autoscaling checklist

Set requests from real profiling data, not guesses. Keep minReplicas at 2+ for HA. Scale up fast, scale down slow. Add a PodDisruptionBudget to every scalable workload. Reserve headroom on nodes for the kubelet and system daemons. Load-test the whole path — pod and node scaling — before you trust it in production.

Next Steps

Continue Learning

Roles and Pod Security.

Next Lesson

Security & RBAC

Additional Resources

Requests and Limits: the Foundation

Pro Tip

Horizontal Pod Autoscaler (HPA)

Scaling on custom and external metrics

Note

Vertical Pod Autoscaler (VPA)

Warning

Cluster Autoscaler: scaling the nodes

Governing Capacity: ResourceQuota and LimitRange

Warning

Common Pitfalls

HPA shows "unknown" for the target metric

Replicas oscillate up and down (flapping)

Nodes never scale down

HPA and VPA both configured, pods thrash

Best Practices

Next Steps