Cloud NetworkingCloud Networking

Part of the Network Engineer Roadmap knowledge base.

Overview

Cloud networking is the practice of designing, building, and operating networks inside a public cloud provider (AWS, Azure, GCP) or across the edge of a provider like Cloudflare. The physical layer — cables, switches, routers, optics, data-center fabric — still exists, but you never touch it. Instead you describe the network you want through an API or Infrastructure-as-Code, and the provider’s software-defined networking (SDN) control plane programs the underlying hardware to match your intent.

For a network engineer coming from the on-prem world, the good news is that the fundamentals do not change: it is still IP addressing (./05-ip-addressing-and-subnetting.md), CIDR, subnets, routing tables (./07-routing.md), NAT, firewalls/ACLs (./11-network-security.md), DNS, VPN/tunnels (./12-vpn-and-tunneling.md), and load balancing. What changes is the abstraction and the operating model: you configure with declarative resources instead of CLI on a physical box, you pay per-use, and responsibility for security and availability is shared between you and the provider.

This note covers:

• How cloud networking differs from on-prem — software-defined, API-driven, elastic, pay-per-use, and the shared responsibility model.
• Virtual networks — VPC/VNet, public vs private subnets, CIDR planning, route tables, internet gateway, NAT gateway, and egress control.
• Connectivity between and into clouds — VPC peering, transit gateway / hub-and-spoke, VPN gateway, dedicated interconnect (Direct Connect / ExpressRoute / Cloud Interconnect), and PrivateLink / private endpoints.
• Cloud security controls — stateful security groups vs stateless network ACLs, cloud firewalls, and WAF.
• Cloud routing & DNS — route propagation, Route 53 / Cloud DNS / Azure DNS, and private hosted zones.
• Load balancing & CDN — L4/L7 cloud load balancers, global vs regional, CDN and anycast.
• Cloudflare — as an edge, security, and DNS provider, plus public resolvers like 1.1.1.1.
• A provider terminology comparison (AWS vs Azure vs GCP).

Deeper, provider-specific notes live under ../../cloud/README.md — see ../../cloud/aws/README.md and ../../cloud/gcp/README.md.

Fundamentals

How cloud networking differs from on-prem

The defining idea is software-defined networking. Your VPC, subnets, and route tables are not physical devices — they are entries in the provider’s distributed control plane that gets pushed to virtualized hosts. This is why a route table change or a new firewall rule takes effect in seconds and why there is no “spanning tree”, no cabling mistakes, and no per-switch config drift in the traditional sense.

The shared responsibility model

Security and availability are split:

• The provider is responsible for the cloud — the physical network, hypervisor, the SDN backbone, DDoS protection at the infrastructure layer, and physical/regional redundancy.
• You are responsible for what is in the cloud — your VPC design, subnet placement, route tables, security groups/NACLs, encryption in transit, key management, IAM for network resources, and how you expose services to the internet.

A misconfigured security group or an over-permissive route is your fault, not the provider’s. This is the number-one source of cloud breaches. See ./11-network-security.md for the security fundamentals that still apply.

Regions, Availability Zones, and edge

• Region — a geographic area (e.g., us-east-1, europe-west1). Each region is isolated for fault tolerance and data residency.
• Availability Zone (AZ) — one or more discrete data centers within a region, with independent power/cooling/networking. Spreading subnets across AZs is how you build highly available architectures.
• Edge / Point of Presence (PoP) — smaller locations worldwide used by CDNs, DNS, and global load balancers to terminate connections close to users.

A VPC is regional (it spans all AZs in a region), while a subnet is tied to a single AZ in AWS (in Azure and GCP the subnet is regional). This has a direct impact on how you lay out CIDR ranges for high availability.

Key Concepts

Virtual networks: VPC / VNet

A VPC (Virtual Private Cloud, AWS/GCP) or VNet (Virtual Network, Azure) is your private, logically isolated slice of the provider’s network. You define its overall address space as one or more CIDR blocks (e.g., 10.0.0.0/16), and everything else lives inside it.

Key properties:

• Uses RFC 1918 private ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16). Pick ranges that will not overlap with your on-prem network or other VPCs you may peer with later — overlapping CIDRs cannot be peered.
• A VPC is isolated by default; nothing routes in or out until you explicitly add gateways and routes.
• You can attach multiple CIDR blocks to grow a VPC that ran out of space.

Subnets: public vs private

A subnet is a slice of the VPC CIDR, and it is where your resources (VMs, containers, load balancers, databases) actually get IP addresses.

• Public subnet — its route table has a default route (0.0.0.0/0) pointing to an internet gateway (IGW). Resources here can have public IPs and be reachable from the internet.
• Private subnet — no direct route to the IGW. Resources have only private IPs and reach the internet (for updates, API calls) via a NAT gateway, or not at all.

The public/private distinction is purely a routing decision — a subnet is “public” because its route table sends 0.0.0.0/0 to an IGW, nothing more. Best practice is to keep application servers and databases in private subnets and only expose load balancers / bastion hosts in public subnets.

VPC 10.0.0.0/16
├── Public subnet  10.0.0.0/24   (AZ-a)  -> route 0.0.0.0/0 to IGW      [ALB, NAT GW]
├── Public subnet  10.0.1.0/24   (AZ-b)  -> route 0.0.0.0/0 to IGW
├── Private subnet 10.0.10.0/24  (AZ-a)  -> route 0.0.0.0/0 to NAT GW   [app servers]
├── Private subnet 10.0.11.0/24  (AZ-b)  -> route 0.0.0.0/0 to NAT GW
├── Data subnet    10.0.20.0/24  (AZ-a)  -> no 0.0.0.0/0 route          [databases]
└── Data subnet    10.0.21.0/24  (AZ-b)  -> no 0.0.0.0/0 route

CIDR planning

CIDR planning in the cloud follows the same subnetting math as on-prem (./05-ip-addressing-and-subnetting.md), with a few cloud-specific rules:

• Leave room to grow. A common pattern is a /16 VPC carved into /24 subnets (256 subnets, 251 usable hosts each after provider reservations).
• Providers reserve addresses in every subnet. AWS reserves the first 4 and the last 1 address of each subnet (network, VPC router, DNS, future use, broadcast). So a /24 gives 251 usable, not 254.
• Never overlap CIDRs across VPCs you intend to peer, or with your on-prem ranges.
• Reserve separate tiers per AZ (public/private/data × 2-3 AZs) so you can build multi-AZ high availability.

Route tables, internet gateway, NAT gateway, egress

• Route table — a set of routes attached to one or more subnets. Each route maps a destination CIDR to a target (IGW, NAT GW, peering connection, transit gateway, VPN gateway, VPC endpoint). The most specific (longest-prefix) match wins, exactly like traditional routing (./07-routing.md).
• Internet gateway (IGW) — a horizontally-scaled, highly available component that lets a VPC send/receive traffic to the internet. It also performs 1:1 NAT for instances that have a public IP.
• NAT gateway — a managed source NAT (PAT) device placed in a public subnet. Private-subnet instances route 0.0.0.0/0 to it so they can make outbound connections (patch, pull images, call APIs) while remaining unreachable from the internet. It is one-way: no unsolicited inbound.
• Egress cost & control — outbound data transfer (egress) to the internet is billed per GB, and cross-AZ / cross-region traffic often costs money too. NAT gateways charge per hour and per GB processed. This makes egress a design and cost concern: use VPC endpoints to keep traffic off the NAT/internet path, and be deliberate about cross-AZ chatter. An egress-only internet gateway is the IPv6 equivalent of a NAT gateway (outbound-only).

Cloud connectivity options

Connecting VPCs to each other, to on-prem, and to services is one of the richest areas of cloud networking.

VPC peering

A 1:1 private connection between two VPCs. Traffic uses private IPs and never touches the internet. Peering is:

• Non-transitive — if A peers B and B peers C, A cannot reach C through B. This is the key limitation that pushes larger designs toward a hub.
• Blocked when CIDRs overlap.
• Simple and cheap for a handful of VPCs, but the mesh explodes (N×(N-1)/2 links) as you add more.

Transit gateway / hub-and-spoke

A transit gateway (AWS Transit Gateway, Azure Virtual WAN / VNet peering with a hub, GCP Network Connectivity Center) is a cloud router that connects many VPCs, VPNs, and Direct Connect/Interconnect links in a hub-and-spoke topology. It is transitive, so every attached VPC can reach every other one through the hub, replacing an unmanageable full mesh of peerings. You control reachability with route tables on the transit gateway and can segment environments (prod/dev) into separate route domains.

VPN gateway

A site-to-site IPsec VPN over the public internet between your on-prem network (or another cloud) and a VPN gateway in the VPC. Encrypted, quick to set up, and cheap — but throughput and latency depend on the internet. Often deployed as a backup to a dedicated link, and with BGP for dynamic route exchange. See ./12-vpn-and-tunneling.md for IPsec/tunneling fundamentals.

Dedicated interconnect (private circuit)

A private, dedicated physical connection from your data center to the provider, bypassing the public internet for consistent bandwidth, lower latency, and lower egress cost at scale:

• AWS Direct Connect
• Azure ExpressRoute
• GCP Cloud Interconnect (Dedicated or Partner)

These are the enterprise choice for predictable, high-volume hybrid connectivity. They typically run BGP and are paired with a VPN for encryption/redundancy.

PrivateLink / private endpoints

PrivateLink (AWS), Private Endpoint / Private Link (Azure), and Private Service Connect (GCP) let you consume a service (a managed service like S3/DynamoDB, or a SaaS/third-party service) over a private IP inside your VPC, so traffic never leaves the provider backbone or traverses the internet. This is the modern way to reach cloud services privately without opening egress or using public endpoints, and it also lets a SaaS vendor expose their app privately to customers.

Cloud security controls

Cloud firewalling happens at two main layers; both build on the ACL/firewall concepts in ./11-network-security.md.

Security groups (stateful) vs network ACLs (stateless)

Rule of thumb: do most of your work with security groups (stateful, easy, per-resource). Use NACLs as a broad subnet backstop or to block specific bad actors. A powerful pattern is to reference another security group as the source (e.g., “allow the app-tier SG to reach the db-tier SG on 5432”) instead of hard-coding IPs.

Azure’s equivalent is the Network Security Group (NSG) (stateful, applies to subnet or NIC); GCP uses VPC firewall rules (stateful, applied by target tags/service accounts at the network level).

Cloud firewalls and WAF

• Managed network firewalls — AWS Network Firewall, Azure Firewall, GCP Cloud Firewall / firewall policies — provide centralized, stateful, L3–L7 inspection, often at the hub, with features like domain filtering and IDS/IPS.
• Web Application Firewall (WAF) — AWS WAF, Azure WAF, Google Cloud Armor, Cloudflare WAF — inspects HTTP(S) at L7 to block SQLi, XSS, bad bots, and enforce rate limits. Deployed in front of load balancers / CDNs. DDoS protection (AWS Shield, Azure DDoS Protection, Cloud Armor, Cloudflare) usually pairs with the WAF/edge.

Cloud routing and DNS

Route propagation

Routes in a route table are either static (you add them) or propagated (learned dynamically). A VPN gateway or transit gateway using BGP can propagate on-prem routes into the VPC route table automatically, so you do not maintain static entries by hand. Longest-prefix match still governs which route is chosen (./07-routing.md).

Cloud DNS

Each provider runs a managed, highly available DNS service:

• AWS Route 53, Azure DNS, Google Cloud DNS.
• Public hosted zones serve your domain to the internet (authoritative DNS), with health checks and routing policies (latency, geolocation, weighted, failover).
• Private hosted zones resolve names only inside your VPC(s) — e.g., db.internal.example.com → a private IP — so internal services find each other without exposing records publicly.
• Providers also give each VPC a built-in resolver (e.g., AWS at the .2 address of the VPC CIDR, the “Amazon-provided DNS”) that resolves both public names and private zones, and can forward to on-prem DNS.

Load balancing and CDN

Cloud load balancers

Cloud load balancers are managed, elastic, and highly available across AZs:

• Layer 4 (network) LB — routes by IP/port, very high throughput, preserves source IP, handles TCP/UDP. E.g., AWS Network Load Balancer (NLB), Azure Load Balancer, GCP passthrough/proxy Network LB.
• Layer 7 (application) LB — routes by HTTP host/path, terminates TLS, supports host/path routing, sticky sessions, WebSockets. E.g., AWS Application Load Balancer (ALB), Azure Application Gateway, GCP HTTP(S) Load Balancer.
• Regional vs global — a regional LB serves one region; a global LB (GCP Global HTTP(S) LB, AWS Global Accelerator, Azure Front Door) uses anycast to send users to the nearest healthy region.

CDN and anycast

A CDN (Content Delivery Network) caches content at edge PoPs close to users, cutting latency and origin load, and absorbing DDoS. Examples: Amazon CloudFront, Azure CDN / Front Door, Google Cloud CDN, and Cloudflare.

Anycast is the routing technique behind global LBs, CDNs, and public DNS: the same IP address is advertised from many locations, and BGP naturally routes each user to the topologically nearest PoP. This gives low latency and inherent DDoS resilience (attacks are spread across many PoPs).

Cloudflare and public resolvers

Cloudflare is a global edge network (a reverse proxy in front of your origin) that combines several of the above functions in one platform:

• Authoritative + managed DNS with very fast propagation.
• CDN caching and WAF / DDoS protection at the edge, on an anycast network.
• Zero Trust / Access, Tunnels (cloudflared) to expose private apps without opening inbound ports.
• Edge compute (Workers) close to users.

Cloudflare also runs the well-known public DNS resolver 1.1.1.1 (and 1.0.0.1), a privacy-focused recursive resolver — comparable to Google Public DNS 8.8.8.8 / 8.8.4.4. These are recursive resolvers anyone can point their devices at; they are not the same as the authoritative hosted zones you run for your own domains.

Provider terminology comparison

The concepts are the same across clouds; only the names differ.

Best Practices

• Plan CIDR before you build. Choose non-overlapping RFC 1918 ranges that leave room to grow and that will not clash with on-prem or future peers. A /16 VPC with /24 subnets per tier per AZ is a safe default.
• Default to private subnets. Put app servers and databases in private subnets; expose only load balancers and (if needed) bastions in public subnets. Reach the internet outbound via NAT gateway, and reach cloud services via PrivateLink/VPC endpoints.
• Least-privilege security groups. Deny by default, open only the exact ports needed, and reference other security groups instead of IP ranges where possible. Use NACLs as a coarse subnet backstop, not your primary control.
• Design for multiple AZs. Spread subnets and load-balancer targets across at least two AZs. Remember a subnet lives in one AZ (AWS) — you need one subnet per AZ per tier.
• Prefer a hub for scale. Two or three VPCs can peer directly; beyond that, use a transit gateway / hub-and-spoke to avoid a non-transitive peering mesh.
• Encrypt and dual-home hybrid links. Use Direct Connect / ExpressRoute / Interconnect for volume and predictability, with a site-to-site VPN as an encrypted backup, and run BGP for dynamic failover.
• Mind egress cost. Outbound and cross-AZ/cross-region data transfer is billed. Use VPC endpoints to keep service traffic off the NAT/internet path, cache with a CDN, and keep chatty components in the same AZ.
• Push security to the edge. Terminate TLS, run a WAF, and enable DDoS protection at the CDN/global-LB layer (CloudFront/Cloudflare/Front Door) in front of your origin.
• Use private DNS zones for internal service discovery instead of hard-coded IPs, and enable DNS resolution/forwarding between cloud and on-prem.
• Everything as code. Manage VPCs, subnets, routes, and firewall rules with Terraform/CloudFormation/Bicep so the network is versioned, reviewable, and reproducible.
• Enable flow logs and monitoring. Turn on VPC Flow Logs (or the equivalent) for traffic visibility, troubleshooting, and security forensics.

For provider-specific depth, see ../../cloud/README.md, ../../cloud/aws/README.md, and ../../cloud/gcp/README.md.

References

• AWS — Amazon VPC User Guide
• AWS — Building a Scalable and Secure Multi-VPC Network Infrastructure
• Microsoft Azure — Virtual Network documentation
• Google Cloud — VPC network overview
• Google Cloud — Cloud Load Balancing overview
• Cloudflare — What is a CDN?
• Cloudflare — What is Anycast DNS?
• Cloudflare — 1.1.1.1 public DNS resolver