Quản lý lưu lượng & QoSTraffic Management & QoS

Part of the Network Engineer Roadmap knowledge base.

Overview

Every link on a network has a finite capacity, and for most of the day that capacity is more than enough. The interesting moments — the ones that make or break a user’s experience — happen during congestion, when more traffic wants to cross a link than the link can carry at that instant. When that happens the router or switch has to make a decision: which packets go first, which packets wait, and which packets get thrown away. Quality of Service (QoS) is the collection of tools that lets you make that decision deliberately, according to business priorities, instead of leaving it to chance.

Without QoS a network is best-effort: every packet is treated identically, first-in-first-out, and during congestion the packets that arrive when the buffer is full are simply dropped. That is perfectly fine for a file download — TCP will retransmit and the user waits a few extra milliseconds. It is a disaster for a voice call, where a 200 ms delay or a 2% loss turns speech into a robotic stutter. On a converged network — one that carries voice, video, and data over the same infrastructure — these traffic types have wildly different needs, and best-effort cannot serve all of them at once.

Traffic management is the broader discipline of controlling how traffic uses the network: not only prioritising during congestion (QoS proper), but also shaping bursts to fit a contracted rate, policing traffic that exceeds a limit, rate-limiting individual users or applications, and steering flows across links. This note builds up from the raw performance metrics you are trying to protect, through the QoS models and mechanisms, to how it all comes together on a real WAN link.

QoS relates closely to other topics in this roadmap. It sits on top of the transport and application protocols in ./06-core-protocols.md (a VoIP call is RTP over UDP; a download is TCP), it complements the redundancy and load-balancing techniques in ./09-redundancy-and-high-availability.md, and the metrics it protects are the same ones you measure with the tools in ./15-tools-and-troubleshooting.md.

Fundamentals

Performance metrics: what QoS actually protects

You cannot manage what you cannot define. The four metrics below are the currency of traffic management — every QoS mechanism exists to improve one or more of them for the traffic that matters.

Bandwidth vs throughput vs goodput

These three are constantly confused, but they are distinct:

The gap between bandwidth and throughput comes from protocol overhead, half-duplex collisions, contention, and the TCP window/RTT relationship. The gap between throughput and goodput comes from headers and retransmissions. When a user says “the internet is slow” they almost always mean goodput feels low, even though the bandwidth they pay for is high.

Latency (delay) and its components

Latency is the time for a packet to travel from source to destination (one-way delay), or there and back (round-trip time, RTT). It is the sum of several independent components, and knowing which one dominates tells you what to fix:

The critical insight: QoS cannot reduce propagation or serialization delay — those are fixed by geography and link speed. What QoS controls is queuing delay, by choosing whose packets wait and whose do not.

Jitter (delay variation)

Jitter is the variation in one-way latency between consecutive packets. If packets in a voice stream arrive 20 ms, 22 ms, 19 ms, 45 ms, 18 ms apart, the jitter is the spread of those inter-arrival times. Real-time applications play out audio/video at a constant rate, so a receiver uses a jitter buffer (de-jitter buffer) to absorb variation — but a buffer that is too small drops late packets, and one that is too large adds delay. Jitter is caused mainly by variable queuing delay, which is exactly why prioritising voice into a low-latency queue reduces jitter.

Packet loss

Packet loss is the percentage of packets that never arrive. Causes include buffer overflow during congestion (tail drop), bit errors on the physical layer, policing drops, and routing changes. TCP recovers from loss by retransmitting (at the cost of goodput and delay); UDP-based real-time media generally does not retransmit — a lost voice packet is simply gone, heard as a click or gap.

Target values for common traffic types

Measuring the metrics

You verify these numbers with the tools covered in ./15-tools-and-troubleshooting.md, briefly:

• ping — RTT and loss, quick reachability check.
• iperf3 — throughput/goodput between two hosts, TCP or UDP; UDP mode reports jitter and loss.
• Speedtest (Ookla/Fast.com) — download/upload throughput and latency to a nearby server, from an end-user perspective.
• mtr — combines traceroute and ping to show per-hop latency and loss, isolating where the problem is.

# Throughput (TCP) between client and a server running `iperf3 -s`
iperf3 -c 10.0.0.1 -t 30

# UDP test at 5 Mbps — reports jitter and packet loss, ideal for VoIP simulation
iperf3 -c 10.0.0.1 -u -b 5M -t 30

Why QoS: converged networks and congestion

Historically voice ran on a dedicated TDM phone network and data ran on IP; each was engineered for its own traffic. Modern networks converge everything onto IP to cut cost and simplify operations — but now a bulk backup can fill the same WAN link as a live sales call. Because these applications have opposite requirements (bulk wants throughput and tolerates delay; voice wants low delay and tolerates loss of throughput), the network needs a way to tell them apart and treat them differently. That is the reason QoS exists.

QoS only does anything during congestion. On an uncongested link every packet goes out immediately regardless of priority — there is nothing to prioritise. The value of QoS is entirely in how it behaves when demand exceeds capacity, which on WAN and internet-facing links happens more often than you would expect, in microbursts even when average utilisation looks low.

Key Concepts

QoS models

There are three architectural approaches to QoS. Real networks usually run DiffServ, but you should understand all three.

• Best-effort is the internet’s default and the baseline everything else improves on.
• IntServ offers hard guarantees but does not scale: a core router carrying millions of flows cannot hold reservation state for each. RSVP survives today mainly as RSVP-TE for MPLS traffic engineering, not for host-level QoS.
• DiffServ (RFC 2474/2475) is the practical winner. Traffic is classified and marked into a handful of classes at the network edge, and each hop applies a Per-Hop Behaviour (PHB) based only on the mark. No per-flow state, so it scales to the whole internet. Almost all enterprise QoS is DiffServ.

Classification and marking

Classification is deciding which class a packet belongs to (by port, protocol, ACL, DPI, or an existing mark). Marking writes that decision into a header field so downstream devices can trust it without re-classifying. The golden rule: classify and mark once, as close to the source as possible (the trust boundary), then trust the mark everywhere else.

Layer 2: CoS (802.1p)

Inside an 802.1Q VLAN tag there is a 3-bit PCP (Priority Code Point) field, commonly called CoS (Class of Service). It gives 8 values (0–7). Because it lives in the VLAN tag, CoS only survives within a Layer 2 domain — it is stripped when a router removes the tag, so it cannot carry priority across a routed WAN.

Layer 3: DSCP (the workhorse)

The IPv4 ToS byte (and IPv6 Traffic Class byte) was redefined by RFC 2474 into a 6-bit DSCP (Differentiated Services Code Point) field (plus 2 bits now used for ECN). 6 bits give 64 values. Because DSCP lives in the IP header, it survives across routed hops end-to-end, which is why it is the primary marking for enterprise QoS.

Standard DSCP values and their PHBs:

AF (Assured Forwarding) defines 4 classes (AF1x–AF4x), each with 3 drop precedences (e.g. AF31 is dropped last, AF33 first within class 3). This lets a class guarantee bandwidth while dropping the least-important packets within it first during congestion. EF (Expedited Forwarding) is the premium PHB: low delay, low jitter, low loss — reserved for voice.

Queuing and scheduling

When packets wait during congestion they sit in queues, and a scheduler decides the order they leave. This is where prioritisation actually happens.

LLQ is the de facto standard for converged networks. Voice goes into the strict-priority queue (served first, so lowest latency and jitter) but that queue is capped so it cannot starve data. Everything else uses CBWFQ classes with guaranteed bandwidth.

Congestion management vs congestion avoidance

These are two different jobs, easy to confuse:

• Congestion management decides how to service the queues once congestion exists — i.e. the scheduling above (LLQ, CBWFQ). It handles the backlog.
• Congestion avoidance tries to prevent the queues from ever filling up, by dropping some packets early to signal senders to slow down before the buffer overflows.

Tail drop (the naive default)

When a queue is full, every new packet is dropped (“dropped off the tail”). Simple, but it causes two problems for TCP:

• Global synchronisation — many TCP flows lose packets at the same instant, all back off together, then all ramp up together, producing a sawtooth of under- and over-utilisation.
• TCP starvation / bursty loss — aggressive flows and fragile flows are punished equally.

WRED (Weighted Random Early Detection)

RED avoids tail drop by randomly dropping packets as the average queue depth grows, before the queue is full. Random early drops hit different flows at different times, so TCP flows back off at staggered moments — no global synchronisation, smoother utilisation. WRED adds weighting by DSCP/drop-precedence: it drops lower-priority (e.g. AF33) packets earlier and more aggressively than higher-priority (AF31) ones. WRED is only sensible for TCP traffic (which responds to loss); you never apply it to the voice/EF queue, which cannot benefit from being told to slow down. ECN is the modern refinement — instead of dropping, the router marks packets so senders slow down without any loss at all.

Traffic shaping vs policing

Both enforce a rate limit, but they handle excess traffic in opposite ways:

The classic use of shaping is when your physical port is 1 Gbps but you only bought 200 Mbps of committed rate (CIR) from the carrier: you shape your egress to 200 Mbps so you buffer and delay the excess, rather than blasting 1 Gbps and letting the carrier’s policer drop it (which would cause retransmissions and worse throughput). Policing is what the carrier uses on the other side, and what you use to cap traffic you do not care about (e.g. guest Wi-Fi).

Token bucket

Both shaping and policing are implemented with a token bucket algorithm:

• Tokens are added to a bucket at the committed rate (CIR), e.g. one token per bit.
• The bucket has a maximum size, the committed burst (Bc) — this allows short bursts above the average.
• When a packet arrives, it needs tokens equal to its size. If enough tokens exist, the packet conforms (sent, tokens removed). If not, it exceeds — a policer drops/re-marks it; a shaper queues it until tokens accumulate.

A dual-rate variant (CIR + PIR, with an excess burst Be) gives three colours — conform (green), exceed (yellow, often re-marked to a higher drop precedence), violate (red, dropped) — the basis of RFC 2698 trTCM policing.

Bandwidth management and rate limiting

Rate limiting is the general act of capping a flow, user, or application to a maximum rate — implemented via policing (drop) or shaping (buffer). Beyond raw QoS classes it is how you:

• Enforce fair use (cap each guest user to 5 Mbps).
• Protect critical services from a runaway backup job.
• Implement API/DDoS protection at the edge (see Cloudflare-style rate limiting).
• Guarantee a minimum via CBWFQ and a maximum via a policer on the same class.

Packet prioritization in practice

Putting the pieces together on a typical WAN edge router: classify at ingress, mark with DSCP, then on the congested egress apply LLQ so EF (voice) gets strict priority, AF classes get guaranteed bandwidth with WRED, and best-effort takes what’s left.

Cisco MQC example (classify → mark → queue → shape)

Cisco QoS uses the Modular QoS CLI (MQC): class-map (classify) → policy-map (what to do) → service-policy (apply to interface).

! 1. Classify traffic into classes
class-map match-any VOICE
 match protocol rtp                 ! or: match dscp ef
class-map match-any CALL-SIGNALING
 match dscp cs3 af31
class-map match-any CRITICAL-DATA
 match access-group name BUSINESS-APPS

! 2. Marking policy at the trust boundary (ingress)
policy-map MARK-INGRESS
 class VOICE
  set dscp ef
 class CALL-SIGNALING
  set dscp cs3
 class CRITICAL-DATA
  set dscp af21
 class class-default
  set dscp default

! 3. Queuing policy for the congested egress (LLQ)
policy-map WAN-EDGE-OUT
 class VOICE
  priority percent 20               ! strict-priority (LLQ), capped at 20%
 class CALL-SIGNALING
  bandwidth percent 5               ! CBWFQ guarantee
 class CRITICAL-DATA
  bandwidth percent 35
  random-detect dscp-based          ! WRED for this TCP class
 class class-default
  bandwidth percent 25
  random-detect                     ! WRED for best-effort
  fair-queue

! 4. Shape total egress to the contracted 200 Mbps, then queue inside it
policy-map WAN-SHAPE
 class class-default
  shape average 200000000
  service-policy WAN-EDGE-OUT       ! nested: queue within the shaped rate

! 5. Apply to interfaces
interface GigabitEthernet0/1
 description LAN
 service-policy input MARK-INGRESS
interface GigabitEthernet0/0
 description WAN to carrier
 service-policy output WAN-SHAPE

Verify with show policy-map interface GigabitEthernet0/0 — it shows per-class packet counts, drops, and queue depths.

Linux tc example (HTB shaping + priority)

On Linux, tc (traffic control) with the HTB (Hierarchical Token Bucket) qdisc does shaping and class-based bandwidth sharing; a priority class handles low-latency traffic.

# Root HTB qdisc on eth0, default class 30
tc qdisc add dev eth0 root handle 1: htb default 30

# Parent class = total shaped rate (200 Mbit)
tc class add dev eth0 parent 1: classid 1:1 htb rate 200mbit

# Voice: guaranteed 20 Mbit, can borrow up to 200, highest priority
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 20mbit ceil 200mbit prio 0
# Critical data: guaranteed 100 Mbit
tc class add dev eth0 parent 1:1 classid 1:20 htb rate 100mbit ceil 200mbit prio 1
# Best-effort default: guaranteed 30 Mbit
tc class add dev eth0 parent 1:1 classid 1:30 htb rate 30mbit ceil 200mbit prio 2

# Give voice a low-latency leaf qdisc; give data a fair/AQM qdisc (fq_codel)
tc qdisc add dev eth0 parent 1:10 handle 10: pfifo
tc qdisc add dev eth0 parent 1:20 handle 20: fq_codel
tc qdisc add dev eth0 parent 1:30 handle 30: fq_codel

# Filter: match DSCP EF (0xb8 in the ToS byte) into the voice class
tc filter add dev eth0 parent 1: protocol ip prio 1 u32 \
   match ip tos 0xb8 0xfc flowid 1:10

# Simple ingress policing: drop anything over 50 Mbit on ingress
tc qdisc add dev eth0 handle ffff: ingress
tc filter add dev eth0 parent ffff: protocol ip u32 match u32 0 0 \
   police rate 50mbit burst 100k drop flowid :1

fq_codel deserves a mention: it is a modern AQM (Active Queue Management) qdisc that fights bufferbloat (excess latency from oversized buffers) by keeping queues short automatically — it is the sensible default leaf qdisc for general traffic.

Traffic management and load balancing

QoS decides how to treat packets on a given link; load balancing decides which link or server a flow uses, spreading load so no single path or server congests in the first place — covered in ./09-redundancy-and-high-availability.md. They are complementary: load balancing reduces the frequency of congestion by using capacity efficiently, while QoS handles congestion gracefully when it still happens. A well-designed WAN uses both — ECMP/load-sharing across redundant links plus per-link QoS policies — and neither is a substitute for the other, nor for simply having enough bandwidth.

Summary table of QoS mechanisms

Best Practices

• Classify and mark once, at the trust boundary. Mark as close to the source as possible (ideally the IP phone or the access switch), then trust the DSCP everywhere else. Re-classifying deep in the network is expensive and error-prone. Establish a trust boundary: untrusted endpoints get re-marked/reset at the access port so users cannot mark their own bulk traffic as EF.
• Use a consistent, standards-based DSCP scheme end-to-end. Follow RFC 4594 class recommendations (EF for voice, AF41 for video, AF31 for signalling, etc.) so your marks mean the same thing at every hop and across carriers. Confirm what DSCP values your ISP/MPLS carrier honours — many remark to BE at the handoff unless you buy a QoS-enabled service.
• QoS is only needed where congestion happens — almost always the WAN edge and internet uplinks, rarely the over-provisioned LAN core. Apply detailed policies there; keep the core simple (just trust and forward).
• Never let the priority (LLQ) queue starve everything else. Always cap it (priority percent) and size it to real voice demand (~33% or less of the link is a common ceiling). Voice above the cap should be dropped by admission control, not allowed to crush data.
• Shape on egress to match your carrier’s rate. If the carrier polices your handoff, shape below their CIR so your router does the buffering (retransmit-free) instead of their policer dropping (retransmit-heavy).
• Only apply WRED to TCP/responsive traffic. Never apply WRED or ECN to the EF/voice queue — UDP real-time media cannot slow down in response to loss.
• Prefer shaping over policing for traffic you care about (buffer, don’t drop); use policing for hard caps on traffic you don’t (guest, scavenger class).
• Fight bufferbloat. Oversized, unmanaged buffers add latency even without a QoS “problem.” Use AQM (fq_codel, CoDel, PIE) on general queues to keep latency low under load.
• Remember QoS manages scarcity, it does not create bandwidth. If a link is chronically saturated, QoS protects voice but data still suffers — the real fix is more capacity or better load balancing (./09-redundancy-and-high-availability.md). QoS buys you graceful degradation, not free capacity.
• Measure before and after. Baseline latency, jitter, and loss with the tools in ./15-tools-and-troubleshooting.md, change one policy at a time, and verify with show policy-map interface / tc -s qdisc show that packets land in the right class and drops fall where you intend.

References

• RFC 2474 — Definition of the Differentiated Services Field (DS Field / DSCP)
• RFC 2475 — An Architecture for Differentiated Services
• RFC 2597 — Assured Forwarding PHB Group & RFC 3246 — Expedited Forwarding PHB
• RFC 4594 — Configuration Guidelines for DiffServ Service Classes
• Cisco — Quality of Service (QoS) Configuration Guides & Design
• Cisco — Comparing Traffic Policing and Traffic Shaping
• Cloudflare — What is Quality of Service (QoS)?
• Linux tc (traffic control) manual pages — tc(8), tc-htb(8), tc-fq_codel(8)