Scaling và AutoscalingScaling and Autoscaling

Part of the Kubernetes Roadmap knowledge base.

Overview

Scaling is the reason most teams adopt Kubernetes in the first place: the promise that capacity follows demand instead of being provisioned for a guessed-at peak. Kubernetes offers scaling at three distinct layers, and confusing them is the single most common source of autoscaling misconfiguration.

• Pod count (horizontal) — run more or fewer copies of a workload. Manual (kubectl scale, replicas:) or automatic via the Horizontal Pod Autoscaler (HPA) and event-driven tools like KEDA.
• Pod size (vertical) — give each Pod more or less CPU/memory. Manual (edit requests/limits) or automatic via the Vertical Pod Autoscaler (VPA).
• Node count (infrastructure) — add or remove machines so Pods have somewhere to land. Automatic via the Cluster Autoscaler or Karpenter.

These layers are complementary and form a chain: the HPA creates more Pods → those Pods are Pending because no node has room → the Cluster Autoscaler/Karpenter notices the unschedulable Pods and adds nodes → the Pods schedule. Understanding this chain is essential, because a common failure mode is “the HPA scaled up but nothing happened” — the real problem being that node scaling was never configured.

The connective tissue across all of it is the resource request. Requests are what the scheduler reserves, what the HPA’s utilization percentage is measured against, and what the Cluster Autoscaler uses to decide whether a Pod fits on a node. Get requests wrong and every layer of autoscaling misbehaves. Requests are the foundation; everything else is built on top.

Fundamentals

Manual scaling

The simplest form of scaling is setting a replica count. Declaratively, it lives in the workload spec:

spec:
  replicas: 4      # Deployment / ReplicaSet / StatefulSet

Imperatively, you can change it live:

kubectl scale deployment/web --replicas=6
kubectl scale statefulset/kafka --replicas=5
kubectl scale --replicas=3 -f deployment.yaml

# Conditional scaling — only if it is currently at a known value (avoids races)
kubectl scale deployment/web --current-replicas=4 --replicas=6

A critical gotcha: if an HPA targets a Deployment, do not also set replicas: in Git. The HPA owns the replica count and will fight your manifest, producing a flapping loop where GitOps resets it to replicas: 3 and the HPA immediately scales it back. The fix is to omit the replicas field entirely from any workload managed by an HPA (or, in Argo CD, ignore differences on that field). On first creation, if you omit replicas, it defaults to 1 until the HPA takes over.

The metrics pipeline

Autoscaling is only as good as the metrics feeding it. Two APIs matter:

• metrics.k8s.io — the resource metrics API, served by metrics-server. Provides live CPU/memory per Pod and node. This powers kubectl top and the HPA’s CPU/memory scaling. metrics-server is not installed by default on vanilla clusters (though managed offerings like GKE and AKS bundle it) — a fresh HPA that reports <unknown>/70% almost always means metrics-server is missing or unhealthy.
• custom.metrics.k8s.io / external.metrics.k8s.io — the custom and external metrics APIs, served by an adapter (Prometheus Adapter, KEDA’s metrics adapter, or a cloud provider’s). These let the HPA scale on application metrics (requests per second, queue depth) rather than raw CPU.

metrics-server samples via the kubelet’s Summary API roughly every 15 seconds. It is designed for autoscaling, not for monitoring or historical data — do not point dashboards at it; use Prometheus for that.

Why requests are the basis of scaling

The HPA’s most common target, averageUtilization, is a percentage of the Pod’s CPU/memory request, not of the node or of a limit. If a container requests 100m CPU and is using 70m, that is 70% utilization regardless of how big the node is.

This has sharp consequences:

• No request set → HPA cannot compute utilization and the metric reports <unknown>. CPU/memory HPAs require a request on the target resource.
• Request set too low → utilization looks artificially high, the HPA over-scales, and you waste Pods.
• Request set too high → utilization looks artificially low, the HPA never triggers, and Pods are throttled without scaling.

The same request drives the Cluster Autoscaler: it simulates whether a Pod’s requests fit on a candidate node. A Pod with no memory request looks free to the autoscaler but can OOM a node in reality. Right-sizing requests is prerequisite work for every autoscaler.

Key Concepts

Horizontal Pod Autoscaler (HPA)

The HPA (autoscaling/v2) periodically (default every 15s, the --horizontal-pod-autoscaler-sync-period) reads metrics and adjusts the replica count of a scalable target (Deployment, ReplicaSet, StatefulSet — anything implementing the scale subresource). The core algorithm is a simple ratio:

desiredReplicas = ceil( currentReplicas × ( currentMetricValue / desiredMetricValue ) )

If 4 replicas average 90% CPU against a 60% target: ceil(4 × 90/60) = ceil(6) = 6 replicas. A ±10% tolerance band prevents thrashing around the target.

Metric types the v2 API supports:

A worked HPA using both CPU and a custom metric, with a scaling behavior block:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70        # % of the CPU *request*
    - type: Pods
      pods:
        metric:
          name: http_requests_per_second
        target:
          type: AverageValue
          averageValue: "500"           # scale to keep ~500 rps per pod
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0     # scale up immediately
      policies:
        - type: Percent
          value: 100                    # at most double
          periodSeconds: 30
        - type: Pods
          value: 4                      # or add at most 4 pods
          periodSeconds: 30
      selectPolicy: Max                 # take whichever allows more
    scaleDown:
      stabilizationWindowSeconds: 300   # wait 5 min of sustained low load
      policies:
        - type: Percent
          value: 20                     # remove at most 20% per minute
          periodSeconds: 60

When multiple metrics are listed, the HPA computes a desired replica count for each and takes the highest — it scales to satisfy the most demanding signal, never dropping below what any metric needs.

The behavior block (v2 only) is what makes production HPAs sane. scaleUp is usually aggressive (react fast to load); scaleDown is deliberately conservative with a stabilization window — the HPA keeps the maximum recommendation seen over the window, so a brief traffic dip doesn’t prematurely kill Pods that will be needed again in seconds. The default scale-down stabilization is 300s.

HPA pitfalls:

• Memory-based HPAs are often a trap: most apps don’t release memory back to the OS, so utilization only ratchets up and the HPA can never scale down. Prefer CPU or a request-rate custom metric for memory-bound apps.
• The HPA and a rolling deployment can interact badly; scaling decisions during a rollout use metrics from a mix of old and new Pods.
• minReplicas: 1 allows single points of failure; production web services should floor at 2–3.

Vertical Pod Autoscaler (VPA)

The VPA adjusts a Pod’s CPU/memory requests (and optionally limits) to match observed usage, solving the “I don’t know what to request” problem. It is a separate installable component (not built in). It has three modes via updatePolicy.updateMode:

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: worker-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: worker
  updatePolicy:
    updateMode: "Auto"
  resourcePolicy:
    containerPolicies:
      - containerName: '*'
        minAllowed: { cpu: 50m, memory: 64Mi }
        maxAllowed: { cpu: 2, memory: 2Gi }
        controlledResources: ["cpu", "memory"]

Historically the VPA in Auto mode had to evict and recreate Pods to change requests, causing disruption — which is why many teams run it in Off mode and apply its recommendations manually or at deploy time. Kubernetes v1.33 graduated in-place Pod resource resize to beta, allowing some resource changes without a restart, which reduces this pain going forward.

The HPA/VPA conflict: never run an HPA and a VPA on the same resource metric. If both watch CPU, the VPA raises the request (lowering utilization %), which makes the HPA think it should scale down, which raises per-Pod load, which makes the VPA raise requests again — an oscillating fight. Safe combinations:

• HPA on CPU + VPA in Off mode (recommendations only).
• HPA on a custom metric (e.g. RPS or queue depth) + VPA on CPU/memory. This is the recommended pattern for scaling both dimensions: HPA handles “how many” from app load, VPA handles “how big” from resource usage.

Cluster Autoscaler

Pod-level autoscaling is useless if there’s nowhere to put the Pods. The Cluster Autoscaler (CA) watches for Pods stuck in Pending due to insufficient resources and grows node groups (AWS ASGs, GCP MIGs, Azure VMSS) to fit them; conversely it removes nodes that have been underutilized for a period and whose Pods can be rescheduled elsewhere.

Key mechanics and caveats:

• It scales on requests, not actual usage. A cluster full of Pods requesting far more than they use will never scale down, and one full of Pods under-requesting can be packed onto too few nodes.
• Scale-down is conservative by design: a node must be under a utilization threshold (default 50%) for a period (default 10 min) and all its Pods must be movable. Pods block eviction if they have no controller, use local storage, have restrictive PodDisruptionBudgets, or carry the cluster-autoscaler.kubernetes.io/safe-to-evict: "false" annotation.
• Scale-from-zero: a node group can scale down to zero nodes and back up. For the CA to know a zeroed group can satisfy a pending Pod’s needs (labels, taints, resources), the group must be tagged so the CA can infer the node shape without a running example.
• It respects PodDisruptionBudgets during scale-down, and honors PriorityClass for pod eviction ordering.
• It is deliberately slow and safe; provisioning a cloud node takes minutes. For faster reaction, over-provision with low-priority “pause” Pods that get preempted by real workloads, buying time while a node spins up.

Karpenter (AWS, and beyond)

Karpenter is a newer, node-provisioning autoscaler (originating at AWS, now a CNCF project) that replaces both the Cluster Autoscaler and node groups. Instead of scaling predefined groups, it looks at the aggregate resource requests of pending Pods and provisions right-sized nodes just-in-time, choosing instance types, sizes, zones, and capacity types (including Spot) directly from the cloud API.

Karpenter’s consolidation actively repacks the cluster onto fewer/cheaper nodes as load drops, which is a major cost win but means more node churn — pair it with PodDisruptionBudgets and graceful shutdown. You describe intent in a NodePool (allowed instance families, zones, limits, disruption policy) rather than sizing groups by hand.

KEDA — event-driven autoscaling

The HPA scales on resource or exposed metrics, but many workloads should scale on events: messages in a queue, lag on a Kafka topic, rows in a database, a cron schedule. KEDA (Kubernetes Event-Driven Autoscaling) fills this gap. It is a CNCF project that acts as a metrics adapter and an HPA generator: you declare a ScaledObject with triggers, and KEDA creates and manages an HPA under the hood, feeding it external metrics from 60+ scalers (SQS, Kafka, RabbitMQ, Prometheus, Azure Service Bus, cron, and more).

KEDA’s headline feature is scale-to-zero: when there are no events, it scales the Deployment to 0 replicas (something a plain HPA cannot do, as its minReplicas floor is 1), then activates the first replica when an event arrives. This is ideal for bursty consumers and batch workers that would otherwise sit idle burning resources.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-consumer
spec:
  scaleTargetRef:
    name: order-consumer          # the Deployment to scale
  minReplicaCount: 0              # scale to zero when idle
  maxReplicaCount: 50
  cooldownPeriod: 300            # wait 5 min of no events before scaling to zero
  pollingInterval: 15
  triggers:
    - type: aws-sqs-queue
      metadata:
        queueURL: https://sqs.eu-west-1.amazonaws.com/123456789012/orders
        queueLength: "20"          # target ~20 messages per replica
        awsRegion: eu-west-1
      authenticationRef:
        name: keda-aws-credentials
    - type: kafka
      metadata:
        bootstrapServers: kafka:9092
        consumerGroup: orders
        topic: orders
        lagThreshold: "100"        # add a replica per 100 messages of lag

With two triggers, KEDA scales to satisfy whichever demands more replicas. Note the distinction: KEDA handles 0↔1 activation itself, then hands 1↔N scaling to the generated HPA. Its ScaledJob variant scales Jobs instead of Deployments, one Job per event batch — ideal for long-running, non-idempotent processing.

Scaling StatefulSets — caveats

StatefulSets can be scaled with kubectl scale or an HPA, but stateful workloads rarely scale as cleanly as stateless ones:

• Ordered, one-at-a-time operations: by default a StatefulSet creates/deletes Pods sequentially (web-0, web-1, web-2…) and scaling down removes the highest ordinal first. This is slow and, for a database, may not be safe.
• Data rebalancing isn’t automatic: adding a replica to Kafka, Cassandra, or Elasticsearch gives you an empty member — the cluster must rebalance partitions/shards, which Kubernetes knows nothing about. Autoscaling replica count without an operator that handles rebalancing can corrupt or unbalance the cluster.
• Scale-down risks data loss: removing a replica can orphan or delete its data. PersistentVolumeClaims are not deleted automatically on scale-down (they’re retained per the persistentVolumeClaimRetentionPolicy), so a naive HPA scaling a database down and up leaves dangling PVCs and under-replicated data.
• Quorum systems have hard limits: consensus systems (etcd, ZooKeeper) need an odd member count for quorum and cannot tolerate arbitrary autoscaling.

Rule of thumb: don’t put an HPA on a StatefulSet backing a stateful system unless a purpose-built operator (e.g. the Strimzi Kafka operator, a database operator) manages membership and rebalancing. For stateful data stores, scale deliberately and manually, or let the operator do it.

Best Practices

1. Set accurate resource requests before enabling any autoscaler. Requests are the denominator for HPA utilization, the fit calculation for the Cluster Autoscaler, and the target for VPA. Wrong requests break every layer — right-size from real usage data (VPA in Off mode or Prometheus) rather than guessing.

2. Install and monitor metrics-server as a prerequisite. CPU/memory HPAs silently report <unknown> without it. Treat it as core infrastructure, run it highly available, and alert on its health.

3. Never set replicas: in Git for an HPA-managed workload. The HPA and your manifest will fight, causing a flapping replica count. Omit the field (and configure your GitOps tool to ignore it).

4. Use autoscaling/v2 and tune the behavior block. Aggressive scaleUp (react fast) plus a conservative scaleDown stabilization window (avoid thrashing) is the difference between a stable and a flapping autoscaler. Default scale-down stabilization is 5 minutes — keep it generous.

5. Prefer CPU or custom metrics over memory for HPA targets. Most runtimes never release memory to the OS, so memory utilization only climbs and the HPA can’t scale down. Scale on CPU or an application signal (RPS, queue depth) instead.

6. Never run HPA and VPA on the same resource metric. They form an oscillating feedback loop. Combine them only as HPA-on-custom-metric + VPA-on-CPU/memory, or run VPA in Off mode for recommendations.

7. Pair Pod autoscaling with node autoscaling. An HPA that creates Pods no node can host just produces Pending Pods. Always deploy the Cluster Autoscaler or Karpenter alongside HPAs, and verify the whole chain end to end.

8. Floor production services at 2–3 replicas with a PodDisruptionBudget. minReplicas: 1 is a single point of failure during node drains and rollouts. A PDB keeps autoscaling and cluster maintenance from taking you to zero available Pods.

9. Use KEDA for event-driven and scale-to-zero workloads. Queue consumers, Kafka processors, and batch jobs should scale on backlog, not CPU, and should idle at zero replicas. Set a sensible cooldownPeriod so brief idle gaps don’t cause churn.

10. Set maxReplicas and node-group limits as cost and blast-radius guardrails. Autoscaling without an upper bound turns a traffic spike (or a metric bug) into a runaway cloud bill. Cap replicas and node counts explicitly.

11. Protect Pods that must not be evicted during scale-down. Use PodDisruptionBudgets and the cluster-autoscaler.kubernetes.io/safe-to-evict: "false" annotation for stateful or long-running Pods so node consolidation doesn’t disrupt them.

12. Configure and tag node groups for scale-from-zero. Add the resource/label/taint tags the Cluster Autoscaler needs to reason about empty groups, so a zeroed group can still be selected for pending Pods.

13. Don’t autoscale StatefulSets backing stateful systems without an operator. Naive replica autoscaling of databases/message brokers risks data loss, dangling PVCs, and quorum failures. Let a purpose-built operator manage membership and rebalancing.

14. Load-test your autoscaling configuration. Autoscaler tuning (thresholds, windows, cooldowns) is only validated under realistic load. Run soak and spike tests, watch the reaction time and steady state, and iterate before trusting it in production.

15. Consider Karpenter for cost-driven, flexible node provisioning on AWS. Just-in-time right-sized nodes and active consolidation cut waste versus fixed node groups — but budget for the extra node churn with PDBs and graceful shutdown.

16. Design apps to shut down gracefully. Autoscaling means Pods are constantly created and destroyed. Handle SIGTERM, drain in-flight work, and set terminationGracePeriodSeconds and a preStop hook so scale-down and consolidation don’t drop requests.

References

• Horizontal Pod Autoscaling
• HPA walkthrough
• Resource metrics pipeline & metrics-server
• Managing resources for containers (requests & limits)
• Vertical Pod Autoscaler
• Resize container resources in-place
• Cluster Autoscaler FAQ
• Karpenter documentation
• KEDA documentation · KEDA scalers
• StatefulSet basics & scaling
• StatefulSet PVC retention policy
• PodDisruptionBudget
• roadmap.sh — Kubernetes