Autoscaling Pods with the HPA

Fixed replica counts waste money at night and fall over at peak. The HorizontalPodAutoscaler (HPA) adds and removes pod replicas based on observed metrics. It needs two things: a source of metrics, and pods with CPU requests, because utilisation is measured as a percentage of the request.

Versions in use on this date. Kubernetes 1.22 with metrics-server 0.5. The HPA API is autoscaling/v1 (CPU only) and autoscaling/v2beta2 (multiple metrics and the behavior field). The autoscaling/v2 version is not stable yet in this release; plan to move manifests when it becomes GA.

The control loop

flowchart TB
  KL[kubelet cAdvisor<br/>pod CPU and memory] --> MS[metrics-server<br/>aggregates every 15 s]
  MS --> MAPI[metrics.k8s.io API]
  HPAC[HPA controller<br/>every 15 s] -->|reads current usage| MAPI
  HPAC -->|desired = ceil current x usage / target| CALC{Compare}
  CALC -->|scale| DEP[Deployment replicas]
  DEP --> POD[Pods]
  POD -->|Pending, no room| CA[Cluster Autoscaler or Karpenter<br/>adds nodes]
  POD --> KL
  EXTRA[Prometheus adapter or KEDA] -.->|custom and external metrics| HPAC

Install metrics-server

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.5.0/components.yaml
kubectl -n kube-system rollout status deploy/metrics-server
kubectl top nodes
kubectl top pods -A --sort-by=cpu | head

EKS, GKE and AKS may include it or provide an add-on. On kind or clusters with self-signed kubelet certificates, add --kubelet-insecure-tls to the container args for lab use only. Pin a release version in your own manifests.

Create an HPA

Quick form:

kubectl -n shop autoscale deployment web --cpu-percent=60 --min=2 --max=10

Declarative, with memory and scaling behaviour (v2beta2):

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata: { name: web, namespace: shop }
spec:
  scaleTargetRef: { apiVersion: apps/v1, kind: Deployment, name: web }
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource: { name: cpu, target: { type: Utilization, averageUtilization: 60 } }
    - type: Resource
      resource: { name: memory, target: { type: AverageValue, averageValue: 200Mi } }
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies: [{ type: Percent, value: 100, periodSeconds: 60 }]
    scaleDown:
      stabilizationWindowSeconds: 300
      policies: [{ type: Pods, value: 1, periodSeconds: 60 }]

The Deployment’s pods must define CPU requests (requests.cpu), or the HPA shows <unknown> for utilisation. If you use an HPA, remove replicas: from the Deployment manifest in Git, otherwise every kubectl apply resets the count.

How the numbers work

desiredReplicas = ceil(currentReplicas × currentMetric / targetMetric). With 4 pods averaging 90% CPU against a 60% target: ceil(4 × 90/60) = 6. Scale-up reacts quickly; scale-down waits for a stabilisation window (5 minutes by default) so a brief lull does not remove capacity. A small tolerance (10%) prevents flapping.

Pick the target well: 60 to 70% CPU leaves headroom while new pods start. Memory-based scaling often misbehaves because many runtimes do not release memory when load drops, so use CPU or request-rate metrics for scaling and memory limits for safety.

Test it

kubectl -n shop get hpa web -w
kubectl run -n shop loadgen --rm -it --image=busybox:1.34 --restart=Never -- /bin/sh -c "while true; do wget -q -O- http://web/; done"
kubectl -n shop describe hpa web            # Events: SuccessfulRescale, conditions
kubectl -n shop get deploy web -w

Better load tests use a tool such as hey or k6 from outside the pod, with realistic requests. Watch latency, not only replica counts: if pods take 90 seconds to become ready, the HPA scales too late, so tune start-up time and readiness probes.

Beyond CPU

• Custom metrics (requests per second, queue depth): expose via the Prometheus Adapter, then use type: Pods or type: Object metrics.
• KEDA scales on event sources (SQS, Kafka lag, cron) and can scale to zero.
• Vertical Pod Autoscaler adjusts requests; do not combine VPA and HPA on the same CPU metric.
• Node capacity: the HPA only creates pods. With no free nodes they sit Pending until the Cluster Autoscaler (or Karpenter) adds nodes; set maxReplicas within what your node limits and budget allow.

Troubleshooting

Symptom	Cause
TARGETS <unknown>/60%	metrics-server missing or unhealthy, or no CPU request on the pods
FailedGetResourceMetric	Pods not ready yet, or the metrics API is failing: kubectl get apiservice v1beta1.metrics.k8s.io
Never scales up	Target too high, maxReplicas reached, or the Deployment is limited by a ResourceQuota
Flapping	Narrow target and short windows; widen scaleDown stabilisation
Scales but latency stays high	Startup too slow, a bottleneck elsewhere (database), or CPU limits throttling