QoS Queuing, Congestion, Policing, Shaping and WRED

When packets wait to leave an interface, queuing decides which goes next. CBWFQ uses weighted round robin to guarantee each class a minimum share of the link; LLQ adds a priority queue the scheduler always serves first, for voice and interactive video. Policing and shaping both measure a rate: a shaper delays the excess by queuing it, a policer discards or re-marks it.

In Daveth Pro (Network)18 min video

By , founder of Daveth · Updated

Opening frame of the video walkthrough of QoS Queuing, Congestion, Policing, Shaping and WRED

Watch this lesson

18 min · Video walkthrough

In Daveth Pro (Network)

The full 18 min walkthrough is in Daveth Pro (Network). See what the plan opens

What this page answers

What is the difference between policing and shaping?

Both compare the measured rate with a configured rate. For a packet that pushes past it, a shaper delays it by queuing, and a policer discards it or marks it differently, chopping the top off the traffic. Read it in “Policing and shaping”

What is LLQ (low latency queuing)?

Queuing that makes one or more queues priority queues: the scheduler always takes the next packet from a priority queue first, so voice waits very little and has little jitter. A policer stops it starving the other queues. Read it in “Always next”

What is CBWFQ?

Class-Based Weighted Fair Queuing: weighted round robin that guarantees each class a minimum share of the bandwidth, set as a percentage of the link, and more when other classes do not need theirs. Read it in “Taking turns”

One line, or several

Friday, month six, week two. Thursday's graph from NMS1 was odd: R3's Gi0/0/1 is a 1 Gbps port, yet its traffic sat flat at 100 Mbps, as if it had hit a ceiling. Ravi rings the ISP and finds the ceiling. The contract is for 100 Mbps, the committed information rate (CIR), and the ISP enforces it: whatever R3 sends above 100 Mbps, the ISP throws away. So R3's own queues never filled. R3 could send at 1 Gbps, and the loss happened at the ISP, which can't tell the meeting from the backup. To choose who waits, R3 has to make the bottleneck its own: send no faster than 100 Mbps itself, and queue there.

Network drawing: Ravi: the meeting, BK1: the backup, shape here, going out, police here, coming in, 1 Gbps link, 100 Mbps contract, Ravi, BK1, HQ2, HQD, R3, ISP, Gi0/0/1

Where the traffic meets the contract: the meeting from Ravi's laptop and the backup from BK1 both reach R3 through HQD; R3's Gi0/0/1 runs at 1 Gbps to the ISP, which polices at 100 Mbps.

Every device queues. It receives a packet, makes its forwarding decision, and if the outgoing interface is busy, holds the packet in an output queue until it's free. Queuing is the set of QoS tools for managing those waiting packets. The simplest is a single queue, first in, first out (FIFO): packets leave in the order they came, so a meeting packet waits behind every backup packet ahead of it. If you've ever ordered at a sandwich shop with one line, you've been in a FIFO queue. (Where it stops: the shop never turns a customer away because the line is long. A full queue does exactly that to the next packet.)

With several queues, the device needs two more parts: a classifier, which chooses the queue for each packet, by its mark or by a fuller match (07.4.1), and a scheduler, which picks the next packet to send when the interface is free. The scheduler is where prioritisation happens: favouring one queue over another.

Taking turns

Round robin takes turns: some from queue 1, then queue 2, then queue 3, then back to 1, taking either one message or a number of bytes from each. Weighted round robin takes more from some queues than from others. Class-Based Weighted Fair Queuing (CBWFQ) uses weighted round robin to guarantee each class a minimum share of the bandwidth, set as a percentage of the link: with 20%, 30% and 50%, queue 1 gets at least 20% of the link when it's busy, and more when the others don't need theirs.

Round robin guarantees shares, not timing. A voice packet that arrives just after the scheduler has left the voice queue waits while every other queue takes its turn, however big the voice queue's share is: that's delay, and delay that varies from packet to packet is jitter. A call needs one-way delay under 150 ms, jitter under 30 ms and loss under 1%, and those needs are the same for every call, even though its bandwidth (about 30 to 320 kbps) depends on the codec. Interactive video has similar needs.

Always next

Low Latency Queuing (LLQ) fixes that. It makes one or more queues priority queues, and the scheduler always takes the next packet from a priority queue first. Packets there wait very little, so jitter is small too, and the queue never fills, so nothing is dropped for lack of room (its policer, coming next, is another matter).

Always-first has a catch. If more traffic arrives for a priority queue than the link can send, the scheduler never gets round to the other queues: queue starvation. So a priority queue is policed, capped at its share, and for a priority queue that share is a maximum, not a minimum; anything beyond it is discarded. Voice and video don't like being discarded either. The tools that stop too many calls from starting in the first place are called Call Admission Control (CAC), and at this level they're only a name.

Put together, the usual strategy for queuing is:

  1. round robin (CBWFQ) for the data classes, and for voice and video that isn't interactive;
  2. when bandwidth is short, much more guaranteed bandwidth for the business-critical data classes than for the rest;
  3. a priority queue (LLQ) for interactive voice and video, for low delay, jitter and loss;
  4. voice in a separate queue from video, so each is policed separately;
  5. enough bandwidth in each priority queue that its policer shouldn't need to discard anything;
  6. Call Admission Control, so that too many calls don't start at once.

Ravi's plan for R3's link to the ISP follows it, using the marks from Thursday. The voice queue carries Sam's trial phone for now; the video queue is for meetings like Thursday's:

QueueMarks (07.4.1)SchedulingShare
voiceEFpriority queue (LLQ), policedat most 10% (10 Mbps)
video meetingsAF41priority queue (LLQ), policedat most 20% (20 Mbps)
business applicationsAF21CBWFQ, round robinat least 30% (30 Mbps) when busy
everything else, the backup includeddefaultCBWFQ, round robinat least 40% (40 Mbps) when busy

A hundred megabits of contract won't carry a hundred and fifty, whatever the marks say.

Policing and shaping

Policing and shaping both watch a rate: they measure the bits per second passing through and compare it with a configured rate. For each packet they ask one question: does this packet push the measured rate past the configured rate? If not, it carries on untouched. If so, a shaper delays it by queuing it, and a policer either discards it or marks it differently.

A policer chops the top off the traffic. It allows a short burst above its rate after a quiet spell, which suits bursty data, and it can work in either direction, though usually on packets coming in. Discarding packets that TCP will only resend looks like it hurts, so why police at all? Because its natural home is the edge between two networks. That's the ISP's side of R3's link: selling 100 Mbps on a 1 Gbps link, the ISP polices what comes in at 100 Mbps, so no customer can take more than they pay for and crowd out the ones who don't, and its reports show a customer when to buy a bigger contract. Instead of discarding the excess, a policer can re-mark it and let it in; further on, if a device is congested, the re-marked packets are candidates to be discarded, and if nothing is congested, they get through anyway.

A shaper is the fix on R3's side, and shapers make sense wherever sending slower than the link could brings a benefit. It works on packets going out: it queues whatever would push the rate over 100 Mbps and releases it at the shaping rate, on its own schedule rather than whenever the interface is free, so the ISP's policer seldom if ever has anything to discard. The order inside R3 is: forward, then shape, then the normal output queue, which, behind a shaper at a tenth of the link's speed, is seldom congested. The price is waiting: shaping adds delay, and probably jitter. Because the shaper holds queues, the queuing tools go inside it: LLQ and CBWFQ, as in Ravi's plan, decide what leaves the shaper first. Here is the same ten seconds of traffic, through the policer:

Policer chart, second by second: traffic above 100 Mbps beyond a 20 Mbit burst allowance is cut off and dropped, 110 Mbit in all.
A policer that discards, second by second: anything above 100 Mbps beyond the 20 Mbit burst allowance is dropped, 110 Mbit in all, and nothing waits.

What the tool proves: with this load, both tools let a 20 Mbit burst through after the quiet first seconds. After that, the policer loses everything above 100 Mbps the moment it arrives, while the shaper holds the excess and sends it once the load falls, dropping only what overflows its queue. A policer holds nothing, so its Waiting column stays at 0. What it doesn't: what TCP does about those losses (the next section), or how big any real queue or burst allowance is; the numbers are an example.