Pod và WorkloadPods and Workloads

Part of the Kubernetes Roadmap knowledge base.

Overview

A Pod is the smallest deployable unit in Kubernetes — not a container, but a wrapper around one or more tightly-coupled containers that share a network namespace, an IP address, and storage volumes. You rarely create bare Pods, though. In practice you declare a higher-level workload controller — a Deployment, StatefulSet, DaemonSet, Job, or CronJob — and let it create and manage Pods for you. The controller is what gives you self-healing, scaling, and rolling updates; the Pod is just the thing that runs.

This layering is deliberate. A Pod on its own is mortal and unmanaged: if it dies, it stays dead, and if its node dies, it’s simply gone. Controllers add the reconciliation loop around Pods — “always keep 3 of these running”, “roll from v1 to v2 without downtime”, “run one on every node”, “run this to completion” — turning fragile individual Pods into a durable, self-maintaining service. Choosing the right controller for a workload is one of the most consequential design decisions in Kubernetes, because it determines how your app scales, updates, survives failure, and holds (or doesn’t hold) state.

A mental model

Think of the Pod template as a stamp and the controller as the machine holding it. You engrave the stamp once (the Pod spec: image, ports, resources, probes) and hand it to a machine that decides how many impressions to make, where, in what order, and what to do when one smudges. A Deployment stamps identical, interchangeable copies. A StatefulSet stamps numbered copies that keep their number and their disk. A DaemonSet stamps exactly one per node. A Job stamps copies that are meant to finish and stop. Same stamp, very different machines — pick the machine to match the workload’s needs.

Fundamentals

The Pod

A Pod groups containers that must run together on the same node and share resources. Within a Pod, containers share:

• The network namespace — one IP per Pod; containers reach each other over localhost and must not collide on ports.
• Storage volumes — volumes declared on the Pod can be mounted into any of its containers, which is how they share files.
• Lifecycle — they’re scheduled, started, and (usually) stopped together.

Most Pods run a single application container. Multi-container Pods exist for helper patterns where a second container is genuinely coupled to the main one:

• Sidecar — a long-running helper alongside the main container: a log shipper, a proxy (service-mesh data plane), a metrics exporter, a file-sync agent. As of Kubernetes v1.28+ sidecars have first-class support (declared as an initContainer with restartPolicy: Always), which fixes startup ordering and shutdown so the sidecar starts before and stops after the app.
• Init containers — run to completion in order, before the app containers start. Use them for setup that must finish first: waiting for a dependency, running a schema migration, fetching config, setting file permissions. If an init container fails, the Pod restarts it (subject to the restart policy) and the app containers never start.
• Ambassador / Adapter — variants of the sidecar that proxy outbound connections or reshape the app’s output for the outside world.

Don’t reach for multi-container Pods to run separate services together — that’s what separate Deployments are for. Co-locate only what shares a lifecycle and a fate.

Pod lifecycle and phases

A Pod moves through a small set of top-level phases (status.phase):

Phases are coarse; the real detail lives in container states (Waiting, Running, Terminated) and conditions (PodScheduled, Initialized, ContainersReady, Ready). Ready is the one that gates traffic — a Pod is added to a Service’s endpoints only when its readiness probe passes.

Restart policy

A Pod’s spec.restartPolicy governs what the kubelet does when a container exits:

• Always (default) — restart the container whenever it stops, success or failure. Used by Deployments/StatefulSets/DaemonSets — long-running services should always be up.
• OnFailure — restart only on non-zero exit. Used by Jobs that should retry until they succeed.
• Never — don’t restart. Used by Jobs where a failed attempt should be recorded, not retried in place.

Restarts use exponential backoff (capped at 5 minutes). A container stuck in a crash-restart-crash cycle shows up as CrashLoopBackOff — a status, not an error type; the real cause is in the logs (--previous) and events. Restart policy applies to containers within a Pod; it does not resurrect a Pod that was deleted or whose node died — that’s the controller’s job.

Probes recap

Three probe types shape a container’s participation in the service (covered in depth in the introduction file, summarized here because they drive workload behavior):

• readinessProbe — “can I serve traffic now?” Failing removes the Pod from Service endpoints without killing it; this is what makes rolling updates safe.
• livenessProbe — “am I wedged?” Failing restarts the container. Never point it at a downstream dependency.
• startupProbe — protects slow starters so a long boot isn’t mistaken for a hang.

Key Concepts

Workload controllers at a glance

ReplicaSet

A ReplicaSet has one job: ensure a specified number of identical Pods are running at all times. It watches Pods matching its selector, creates more if there are too few, deletes some if there are too many. You almost never create ReplicaSets directly, because they have no update strategy — changing the Pod template of a ReplicaSet does not roll out to existing Pods. Instead you use a Deployment, which manages ReplicaSets for you and adds rollout logic on top.

Deployment — rollouts, strategies, and rollback

A Deployment is the default workload for stateless apps. It manages ReplicaSets to provide declarative updates: change the Pod template (e.g. bump the image) and re-apply, and the Deployment orchestrates the transition from the old ReplicaSet to a new one.

Two strategies:

• RollingUpdate (default) — gradually replace old Pods with new ones, controlled by maxSurge (how many extra Pods above desired count may exist during the roll) and maxUnavailable (how many below desired may be missing). This gives zero-downtime deploys when readiness probes are set. maxSurge: 1, maxUnavailable: 0 is a safe, conservative default: always add one new Pod and wait for it to be Ready before removing an old one.
• Recreate — terminate all old Pods, then create new ones. Causes downtime, but is required when old and new versions can’t run simultaneously (e.g. an exclusive lock, or an incompatible schema change).

Because each version is a distinct ReplicaSet that the Deployment keeps around (up to revisionHistoryLimit), rollback is trivial:

kubectl rollout status deploy/web            # watch the roll
kubectl rollout history deploy/web           # list revisions
kubectl rollout undo deploy/web              # back to the previous revision
kubectl rollout undo deploy/web --to-revision=3
kubectl rollout restart deploy/web           # re-roll (e.g. to pick up a changed Secret)
kubectl rollout pause deploy/web             # pause mid-roll (for canary-style checks)

StatefulSet — stable identity and ordered lifecycle

Some workloads can’t be interchangeable, throwaway Pods — a database replica needs to keep the same name, the same storage, and the same network identity across restarts and reschedules. That’s a StatefulSet. It provides:

• Stable, ordinal names — web-0, web-1, web-2, not random hashes. A Pod that dies comes back with the same name.
• Stable per-Pod storage — via volumeClaimTemplates, each Pod gets its own PersistentVolumeClaim (data-web-0, data-web-1, …) that follows it across reschedules. Deleting the StatefulSet does not delete these PVCs, protecting the data.
• Ordered, graceful deployment and scaling — Pods are created in order (0, then 1, then 2), each waiting for the previous to be Ready, and terminated in reverse order. This matters for clustered systems that elect leaders or join members one at a time.
• A stable network identity via a headless Service (clusterIP: None), which gives each Pod its own DNS name: web-0.web.default.svc.cluster.local. Clients (and peers) address individual replicas directly rather than a load-balanced VIP.

StatefulSets are more operationally involved than Deployments; for complex stateful systems, a purpose-built Operator (a CRD plus a controller, e.g. the CloudNativePG, Strimzi/Kafka, or Zalando Postgres operators) usually manages the StatefulSet for you and handles backups, failover, and version upgrades.

DaemonSet

A DaemonSet ensures one copy of a Pod runs on every node (or every node matching a selector). As nodes join the cluster, the DaemonSet controller adds the Pod to them; as nodes leave, their Pods are garbage-collected. This is the model for node-level infrastructure: log collectors (Fluent Bit), metrics agents (node-exporter), CNI plugins, CSI drivers, and security agents. DaemonSet Pods often need tolerations to run on tainted nodes (including control-plane nodes) and frequently mount host paths, so treat them as privileged infrastructure and review them carefully.

Job and CronJob

A Job runs Pods until a specified number complete successfully, then stops — the model for finite, run-to-completion work (a migration, a batch computation, a data export). Key knobs: completions (how many successful runs are needed), parallelism (how many run at once), backoffLimit (retries before the Job is marked Failed), and activeDeadlineSeconds (wall-clock timeout). Pair with restartPolicy: OnFailure or Never.

A CronJob creates Jobs on a schedule (standard cron syntax), for backups, reports, and periodic cleanup:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: nightly-backup
spec:
  schedule: "0 2 * * *"                 # 02:00 every day
  concurrencyPolicy: Forbid             # don't start a run if the last is still going
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 3
  jobTemplate:
    spec:
      backoffLimit: 2
      template:
        spec:
          restartPolicy: Never
          containers:
            - name: backup
              image: myorg/pgbackup:1.2
              args: ["--target", "s3://backups/db"]

Watch out for concurrencyPolicy (Allow/Forbid/Replace) so slow runs don’t pile up, and startingDeadlineSeconds so a missed schedule (controller downtime) doesn’t fire a storm of catch-up Jobs.

Labels, selectors, and annotations

Labels (key: value on metadata) are how controllers claim Pods and Services find endpoints — the loose-coupling glue. Adopt the recommended app.kubernetes.io/name, app.kubernetes.io/instance, app.kubernetes.io/version, and app.kubernetes.io/component labels so dashboards, cost tools, and selectors all speak the same language. Selectors come in two forms: equality-based (app=web) and set-based (environment in (prod, staging)). A controller’s spec.selector is immutable on Deployments/StatefulSets — pick it carefully, because you can’t change it later without recreating the object.

Annotations carry non-identifying metadata that tools and controllers read — build info, a config hash to force rollouts, ingress-controller settings, kubectl.kubernetes.io/last-applied-configuration. You never select on annotations, and they can hold large or structured values that labels can’t.

Pod Disruption Budgets

Disruptions come in two kinds: involuntary (node crash, hardware failure — you can’t prevent these) and voluntary (node drain for an upgrade, cluster scale-down, kubectl drain). A PodDisruptionBudget (PDB) protects availability during voluntary disruptions by capping how many Pods of a set may be down at once:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: web
spec:
  minAvailable: 2          # or: maxUnavailable: 1
  selector:
    matchLabels:
      app: web

With this, a node drain will block evicting a web Pod if doing so would drop below 2 available, forcing the drain to wait until a replacement is Ready elsewhere. A PDB is meaningless with a single replica — you need enough replicas to have slack. Note a PDB does not protect against involuntary disruptions; it only makes voluntary ones respect your availability floor.

Choosing the right workload type

• Stateless app (web server, API, worker that keeps no local state) → Deployment.
• Stateful, needs stable identity/storage (database, message broker, quorum system) → StatefulSet (or an Operator that manages one).
• Runs on every node (agent, driver, collector) → DaemonSet.
• Finite task that must complete (migration, batch job) → Job.
• Runs on a schedule → CronJob.
• Just need N copies, no rollout logic → still a Deployment (never a bare ReplicaSet).

Common mistake: forcing a database into a Deployment. Deployments treat Pods as interchangeable and use shared or ephemeral storage, so two rolling Pods can end up fighting over the same volume or losing data. Stateful data belongs in a StatefulSet with per-Pod PVCs, or better, a managed database.

Worked example — Deployment and StatefulSet, explained

A stateless frontend as a Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: storefront
  labels:
    app.kubernetes.io/name: storefront
spec:
  replicas: 4
  selector:
    matchLabels:
      app.kubernetes.io/name: storefront
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1                 # add one extra Pod during a roll
      maxUnavailable: 0           # never drop below desired capacity
  template:
    metadata:
      labels:
        app.kubernetes.io/name: storefront
    spec:
      containers:
        - name: app
          image: ghcr.io/acme/storefront:2.5.1     # immutable tag
          ports:
            - containerPort: 8080
          resources:
            requests: { cpu: 100m, memory: 128Mi } # drives scheduling
            limits:   { cpu: 500m, memory: 256Mi } # caps usage
          readinessProbe:                          # gates traffic during rollouts
            httpGet: { path: /healthz, port: 8080 }
          livenessProbe:
            httpGet: { path: /livez, port: 8080 }
            initialDelaySeconds: 10

Why these choices: interchangeable replicas → Deployment; maxUnavailable: 0 keeps full capacity during the roll; the readiness probe means a new Pod receives traffic only once healthy, and the roll pauses if new Pods never become Ready — a broken image can’t take the service down. Resource requests let the scheduler place Pods and enable autoscaling.

A PostgreSQL replica set as a StatefulSet (schematic — use an operator in production):

apiVersion: v1
kind: Service
metadata:
  name: pg                       # headless Service → per-Pod DNS
spec:
  clusterIP: None
  selector:
    app: pg
  ports:
    - port: 5432
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: pg
spec:
  serviceName: pg                # ties the StatefulSet to the headless Service
  replicas: 3
  selector:
    matchLabels:
      app: pg
  template:
    metadata:
      labels:
        app: pg
    spec:
      terminationGracePeriodSeconds: 30
      containers:
        - name: postgres
          image: postgres:16.3
          ports:
            - containerPort: 5432
          volumeMounts:
            - name: data
              mountPath: /var/lib/postgresql/data
          readinessProbe:
            exec: { command: ["pg_isready", "-U", "postgres"] }
  volumeClaimTemplates:          # each Pod gets its OWN persistent disk
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 20Gi

Why these choices: the Pods are not interchangeable — pg-0, pg-1, pg-2 keep their names and each owns a private data-pg-N PVC that survives reschedules and even deletion of the StatefulSet. The headless Service (clusterIP: None) gives every Pod a stable DNS name (pg-0.pg...) so clients and replicas can address specific members — essential for primary/replica topologies. Pods come up in order, so pg-0 (say, the primary) is Ready before replicas join. This is exactly what a Deployment cannot provide.

Best Practices

1. Never run bare Pods in production — always use a controller. A lone Pod has no self-healing; a Deployment/StatefulSet/DaemonSet reschedules it after node loss.
2. Match the controller to the workload. Stateless → Deployment, stateful → StatefulSet (or Operator), per-node → DaemonSet, finite → Job, scheduled → CronJob. Forcing a database into a Deployment risks data loss.
3. Set resource requests and limits on every container. Requests drive scheduling and autoscaling; limits cap blast radius. Missing requests make the scheduler blind; missing limits let one Pod starve a node.
4. Define readiness (and where useful, liveness/startup) probes. Readiness makes rolling updates zero-downtime by keeping unready Pods out of the load balancer; a rollout with good probes self-arrests on a bad image.
5. Default to RollingUpdate with maxUnavailable: 0, and only use Recreate when versions genuinely can’t coexist. Keep revisionHistoryLimit sane so rollback works but old ReplicaSets don’t accumulate forever.
6. Run at least 2–3 replicas and pair them with a PodDisruptionBudget. This survives node drains and cluster upgrades without dropping to zero — a PDB on a single-replica app does nothing.
7. Use init containers for ordered setup, sidecars for coupled helpers — and prefer the native sidecar (initContainer with restartPolicy: Always) so startup/shutdown ordering is correct. Don’t co-locate unrelated services in one Pod.
8. Give StatefulSets a headless Service and volumeClaimTemplates. That’s what delivers stable identity and per-Pod storage; without them you don’t get StatefulSet semantics.
9. Handle SIGTERM and tune terminationGracePeriodSeconds with a preStop hook so in-flight requests drain during rollouts and drains instead of being cut off.
10. Label consistently with the app.kubernetes.io/* convention, and treat controller spec.selector as immutable — choose it deliberately, because changing it means recreating the object.
11. Use annotations, not labels, for non-selector metadata (build info, config hashes, tool settings); reserve labels for things you actually select or group by.
12. For Jobs, set backoffLimit and activeDeadlineSeconds; for CronJobs, set concurrencyPolicy and history limits so retries are bounded and slow runs don’t pile up.
13. Scale stateless apps with an HPA, not by editing replicas by hand, and never run an HPA and VPA on the same metric — they fight.
14. Review DaemonSets as privileged infrastructure. They often tolerate taints and mount host paths on every node; a bad one has cluster-wide reach.
15. Prefer a managed service or a mature Operator for complex stateful systems. Hand-rolling database failover on a raw StatefulSet is far more work than it looks.
16. Watch CrashLoopBackOff as a symptom, not a cause. The real reason is in kubectl logs --previous and kubectl describe events (bad config, missing dependency, OOM) — fix that, not the restart loop.

References

• Pods · Pod lifecycle
• Init containers · Sidecar containers
• Workloads overview
• ReplicaSet
• Deployments
• StatefulSets · StatefulSet basics tutorial
• DaemonSet
• Jobs · CronJob
• Labels and selectors · Recommended labels · Annotations
• Pod Disruption Budgets · Specifying a Disruption Budget
• Configure liveness, readiness and startup probes
• Horizontal Pod Autoscaling
• Book: Kubernetes Up & Running (Hightower, Burns, Beda — O’Reilly)
• roadmap.sh — Kubernetes