ServerlessServerless

Part of the DevOps Roadmap knowledge base.

Overview

Serverless is a cloud execution model where the provider fully manages server provisioning, scaling, and maintenance, and you pay only for the compute actually consumed. “Serverless” doesn’t mean there are no servers — it means you don’t manage them. The best-known form is Functions as a Service (FaaS): small units of code triggered by events, exemplified by AWS Lambda, Azure Functions, Google Cloud Functions, and Cloudflare Workers. A newer branch, serverless containers (Google Cloud Run, AWS Fargate, Azure Container Apps), applies the same pay-per-use, scale-to-zero model to full container images.

For DevOps, serverless shifts the operational burden dramatically: no OS patching, no capacity planning, no idle servers. Scaling from zero to thousands of concurrent executions is the platform’s job. In exchange, you accept constraints — execution time limits, cold starts, vendor-specific APIs — and a different cost and debugging model.

Serverless shines for event-driven glue code, APIs with spiky or unpredictable traffic, and asynchronous processing pipelines. It’s a poor fit for long-running, latency-critical, or extremely high-volume steady workloads, where dedicated compute is cheaper and more predictable.

The two things “serverless” actually promises

Beneath the marketing, serverless makes two concrete promises that define the category:

1. Scale to zero. When no requests arrive, you run nothing and pay nothing. A traditional VM or Kubernetes pod is always on, always billing. This is the property that makes serverless economically magical for idle-heavy workloads and economically terrible for constantly-busy ones.
2. Automatic, request-proportional scaling. The platform adds capacity as load rises and removes it as load falls, with no autoscaling policy for you to tune. You never provision for peak; the platform does it per-request.

Everything else — the constraints, the cost model, the cold starts — flows from these two promises. When someone asks “is this workload a good fit for serverless?”, they are really asking “does this workload benefit from scaling to zero and per-request elasticity more than it suffers from the constraints those properties impose?”

Fundamentals

FaaS in a nutshell

1. You write a function (a handler) and declare its trigger (HTTP request, queue message, file upload, schedule, database change).
2. The platform packages, deploys, and runs it on demand — one instance per concurrent event by default.
3. You’re billed per invocation and per compute time (GB-seconds), typically with a generous free tier.

# AWS Lambda handler (Python)
import json

# Runs ONCE per cold start — reuse expensive setup across warm invocations
db_client = create_db_client()

def handler(event, context):
    name = event.get("queryStringParameters", {}).get("name", "world")
    return {
        "statusCode": 200,
        "body": json.dumps({"message": f"Hello, {name}!"})
    }

The key structural insight is in the comment: the handler runs on every invocation, but code outside the handler runs only on a cold start. Putting expensive initialization (DB clients, SDK objects, config loading) in global scope means warm invocations reuse it for free — one of the highest-leverage optimizations in serverless.

Major platforms

Two families, different trade-offs. The FaaS family (Lambda, Cloud/Azure Functions, Workers) optimizes for tiny event-driven units and the finest-grained billing. The serverless-container family (Cloud Run, Fargate, Container Apps) optimizes for portability (any language/binary, no framework lock-in) and for one instance serving many concurrent requests. Workers are a third thing entirely: edge isolates that trade the full Node/OS environment for near-zero cold start and global distribution.

Event-driven architecture

Serverless functions are natural consumers in event-driven systems:

• Synchronous (request/response): API Gateway → Lambda → response (client waits). Errors surface directly to the caller; there is no automatic retry.
• Asynchronous (fire-and-forget): S3 upload event → Lambda resizes image. The platform queues the event and retries automatically on failure (Lambda retries twice by default), with a dead-letter queue for poison messages.
• Stream/queue-based (poll): SQS/Kinesis/Pub/Sub → function processes batches; the platform manages polling, batching, checkpointing, and ordering guarantees (per-shard for Kinesis, per-message-group for FIFO SQS).

These three invocation types behave differently on failure, retries, and ordering — a distinction that trips up newcomers. Reaching for the wrong one (e.g., calling a slow downstream synchronously when it should be queued) is a common design mistake.

Event-driven design decouples producers from consumers: services communicate through events, scale independently, and failures are isolated and retryable. Orchestration tools stitch functions into workflows: AWS Step Functions, Azure Durable Functions, and Google Workflows let you express retries, branching, parallelism, and human-approval steps declaratively instead of chaining functions by hand.

Cold starts

A cold start happens when the platform must create a new execution environment: allocate a sandbox, download code, start the runtime, and run your initialization code. It adds latency (tens of milliseconds to several seconds) to the first request served by that instance. Subsequent requests to the same warm instance skip all of this.

The lifecycle: cold start → warm invocations → idle → reclaimed. There is no way to pin an instance warm forever on consumption plans; the platform reclaims idle instances to free capacity. This is why steady low traffic can paradoxically feel slower than steady high traffic — low traffic means instances go idle and get reclaimed between requests.

When cold starts do and don’t matter. For asynchronous and batch work, cold starts are irrelevant — nobody is waiting. For internal APIs with steady traffic, instances stay warm and cold starts are rare. Cold starts only bite on user-facing, latency-sensitive, spiky paths — and that’s exactly where provisioned concurrency or an edge runtime earns its keep. Don’t pay for provisioned concurrency on a nightly batch job.

Key Concepts

Cost model

You pay for requests + duration × memory, billed in GB-seconds:

AWS Lambda (us-east-1, x86, indicative):
  $0.20 per 1M requests
  $0.0000166667 per GB-second

Example: 5M requests/month, 512 MB, 200 ms average
  Requests: 5M × $0.20/1M                     = $1.00
  Compute:  5M × 0.2s × 0.5 GB × $0.0000167   ≈ $8.33
  Total ≈ $9.33/month (before free tier)

Now contrast the same code at high, constant volume:

Same function at 500 req/s sustained (≈1.3B requests/month), 200 ms, 512 MB:
  Requests: ~1.3B × $0.20/1M                  ≈ $260
  Compute:  ~1.3B × 0.2s × 0.5 GB × $0.0000167 ≈ $2,167
  Total ≈ $2,400/month

A pair of always-on containers/VMs handling the same steady load
could cost a fraction of that.

This is the crossover that defines serverless economics: cheap when idle-heavy and spiky, expensive when busy and steady. Somewhere around “consistently busy a large fraction of the day,” a right-sized container or reserved VM becomes cheaper. Model it before committing.

Watch the hidden costs that don’t show up in the per-invocation math: API Gateway per-request fees (often larger than the Lambda bill itself), CloudWatch/log ingestion and storage, NAT Gateway charges for VPC-attached functions reaching the internet, cross-service data transfer, and Step Functions state transitions. The compute line is frequently not the biggest number on a serverless bill.

Concurrency: the number that actually governs behavior

Serverless scaling is best understood through concurrency — how many instances run at once.

• FaaS default: one instance handles exactly one request at a time. 1,000 simultaneous requests → 1,000 instances. This is simple to reason about but multiplies downstream pressure (see the database problem below).
• Cloud Run / Container Apps: one instance can handle many concurrent requests (configurable, e.g., --concurrency=80). For I/O-bound APIs this is dramatically cheaper — one instance serves 80 requests instead of spinning up 80.
• Reserved / maximum concurrency: cap how far a function can scale to protect downstream systems (a database, a rate-limited third-party API) from being overwhelmed by a scaling storm.

The mental model: serverless removes your scaling policy but not your scaling consequences. The platform will happily scale you to 10,000 concurrent instances, and all 10,000 will try to open a database connection at once.

Example: Lambda + API Gateway flow

Client ──HTTPS──▶ API Gateway ──invoke──▶ Lambda ──▶ DynamoDB
                     │                       │
                     └── auth (Cognito/JWT)  └── logs/traces → CloudWatch/X-Ray

Deploying the whole stack with AWS SAM:

# template.yaml (AWS SAM)
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
  HelloFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: app.handler
      Runtime: python3.12
      MemorySize: 256
      Timeout: 10
      Events:
        Api:
          Type: HttpApi
          Properties:
            Path: /hello
            Method: GET

sam build && sam deploy --guided     # package + deploy
sam local invoke HelloFunction       # test locally
sam logs -n HelloFunction --tail     # stream logs

Deployment framework landscape: AWS SAM and the Serverless Framework are FaaS-focused and terse; AWS CDK and Pulumi give you general-purpose languages; Terraform/OpenTofu is the cloud-agnostic default when serverless is one part of a larger estate. For Cloud Run and container-based serverless, a plain Dockerfile plus gcloud run deploy or Terraform is usually enough.

Serverless containers

Cloud Run is the cleanest example: give it any container listening on $PORT, and it handles TLS, autoscaling (including to zero), revisions, and traffic-splitting for canary releases.

gcloud run deploy my-api \
  --image=gcr.io/my-project/my-api:1.4.0 \
  --region=asia-southeast1 \
  --min-instances=0 --max-instances=20 \
  --concurrency=80 \
  --cpu=1 --memory=512Mi \
  --allow-unauthenticated

Compared to FaaS, serverless containers offer:

• No framework lock-in — any language, any binary, any dependency; if it runs in a container, it runs here.
• High concurrency per instance — one instance serves many requests, so I/O-bound APIs cost far less than one-request-per-instance FaaS.
• Portability — the same image runs on your laptop, in Cloud Run, and in Kubernetes, so migrating off is a redeploy, not a rewrite. This dramatically reduces lock-in versus FaaS, whose handler signatures and event formats are vendor-specific.
• A gentler local-dev story — docker run reproduces production far more faithfully than emulating a FaaS event source.

The trade-off is that you own the container (base image, patching, size) and you don’t get the deep native event-source integrations that make Lambda so powerful for glue code.

When serverless fits — and when it doesn’t

Good fit

• Spiky, unpredictable, or low-average traffic (pay nothing when idle)
• Event-driven glue: file/image processing, webhooks, notifications, scheduled/cron jobs, S3-triggered pipelines
• Rapid prototyping and small teams without ops capacity
• Burst parallelism (fan out 1,000 workers for a batch job, then scale to zero)
• Backends for JAMstack sites and mobile apps with variable usage
• Automation and “cloud plumbing” — reacting to infrastructure events, ChatOps, scheduled cleanup

Poor fit

• Long-running processes (> 15 min on Lambda), massive-scale websockets, stateful servers
• Ultra-low-latency paths that can’t tolerate cold starts (unless using edge runtimes or provisioned concurrency)
• High, steady 24/7 load — reserved VMs/containers become materially cheaper
• Workloads needing GPUs, very large memory, or specialized hardware (though this is slowly changing)
• Heavy, chatty database workloads that would exhaust connection pools under FaaS’s one-request-per-instance model
• Teams that require full portability — FaaS APIs, triggers, and event formats are vendor-specific (serverless containers are the escape hatch here)

The honest summary: serverless is not a default and not a fad — it’s a tool whose sweet spot is event-driven work and variable traffic where operational simplicity is worth accepting constraints. Reach for it there; reach for containers/Kubernetes for steady high-throughput services; and don’t let “serverless-first” or “serverless-never” dogma make the decision for you.

Best Practices

1. Keep functions small and single-purpose — one trigger, one responsibility; easier to test, deploy, secure, and reason about failure modes than a “Lambdalith” that does everything.
2. Make handlers idempotent — async and stream triggers retry on failure, and at-least-once delivery means duplicates happen; processing the same event twice must be safe (use idempotency keys, conditional writes, or dedup tables).
3. Initialize outside the handler — create DB clients, SDK objects, and load config in global scope so warm invocations reuse them; this cuts both latency and cost. Never open a fresh connection per invocation.
4. Set explicit timeouts and memory, and measure — right-size both; on Lambda more memory also means proportionally more CPU, so a bump can make a function finish faster and cost less. Use AWS Lambda Power Tuning to find the sweet spot.
5. Use dead-letter queues (DLQs) for async events — never silently drop failed events; route them somewhere inspectable and replayable, and alert on DLQ depth.
6. Avoid synchronous function-to-function chains — you pay for both functions’ wall time, multiply failure modes, and compound cold-start latency; use queues, Step Functions, Durable Functions, or an event bus instead.
7. Store all state externally — functions are ephemeral and share nothing; use DynamoDB/Redis/S3 for state and Step Functions/Durable Functions for workflow state. Never rely on local disk or in-memory state surviving between invocations.
8. Keep deployment packages lean — smaller artifacts mean faster cold starts; trim dependencies, tree-shake, split heavy deps into layers, or move to a slim container image.
9. Guard downstream systems against scaling storms — a function scaling to thousands of instances will exhaust a relational database’s connections; use reserved/maximum concurrency, RDS Proxy (or equivalent connection pooler), and queues as buffers between the elastic tier and the fixed-capacity tier.
10. Instrument everything with structured logs and tracing — correlation IDs and distributed tracing (X-Ray, OpenTelemetry, Application Insights) are essential when a single request crosses API Gateway, a function, a queue, and three managed services; without them, debugging is guesswork.
11. Manage cold starts deliberately, not reflexively — use provisioned concurrency / min-instances only on user-facing latency-sensitive paths; leave batch and async work on plain on-demand where cold starts don’t matter.
12. Apply least-privilege IAM per function — give each function its own role scoped to exactly the resources it touches; a compromised or buggy function should not be able to read every table in the account.
13. Deploy via IaC and CI/CD — SAM, Serverless Framework, CDK, Pulumi, or Terraform; never hand-edit function code in the console, or your deployed state drifts from source control and becomes unreproducible.
14. Watch the whole bill, not just compute — account for API Gateway, log ingestion/retention, NAT Gateway, and inter-service transfer; set log retention (raw logs are a silent, unbounded cost) and budget alerts.
15. Model costs before committing and re-evaluate as traffic grows — estimate requests × duration × memory versus a small always-on container; the right answer changes as a workload matures from spiky prototype to steady production service.
16. Prefer serverless containers when portability or high concurrency matters — Cloud Run / Container Apps / Fargate keep you off vendor-specific handler APIs and let one instance serve many requests, which is both cheaper for I/O-bound APIs and easier to migrate.

References

• roadmap.sh — DevOps Roadmap
• AWS Lambda Documentation
• AWS Lambda operator guide — performance optimization
• AWS Lambda — Provisioned Concurrency
• AWS Lambda Power Tuning
• Azure Functions Documentation
• Azure Durable Functions
• Azure Container Apps Documentation
• Google Cloud Run Documentation
• Google Cloud Functions Documentation
• Google Cloud Run — Cold start and concurrency
• Cloudflare Workers Documentation
• AWS Serverless Application Model (SAM)
• AWS Step Functions Documentation
• Serverless Framework Documentation
• Serverless Land (AWS patterns and examples)
• Martin Fowler — Serverless Architectures (Mike Roberts)
• CNCF — Serverless Whitepaper