QoS in Linux with TC and Filters

Phil Sutter (phil@nwl.cc)

January 2016

Standard practice when transmitting packets over a medium which may block (due to congestion, e.g.) is to use a queue which temporarily holds these packets. In Linux, this queueing approach is where QoS happens: A Queueing Discipline (qdisc) holds multiple packet queues with different priorities for dequeueing to the network driver. The classification (i.e. deciding which queue a packet should go into) is typically done based on Type Of Service (IPv4) or Traffic Class (IPv6) header fields but depending on qdisc implementation, might be controlled by the user as well.

Qdiscs come in two flavors, classful or classless. While classless qdiscs are not as flexible as classful ones, they also require much less customizing. Often it is enough to just attach them to an interface, without exact knowledge of what is done internally. Classful qdiscs are the exact opposite: flexible in application, they are often not even usable without insightful configuration.

As the name implies, classful qdiscs provide configurable classes to sort traffic into. In it's basic form, this is not much different than, say, the classless pfifo_fast which holds three queues and classifies per packet upon priority field. Though typically classes go beyond that by supporting nesting and additional characteristics like e.g. maximum traffic rate or quantum.

When it comes to controlling the classification process, filters come into play. They attach to the parent of a set of classes (i.e. either the qdisc itself or a parent class) and specify how a packet (or it's associated flow) has to look like in order to suit a given class. To overcome this simplification, it is possible to attach multiple filters to the same parent, which then consults each of them in row until the first one accepts the packet.

Before getting into detail about what filters there are and how to use them, a simple setup of a qdisc with classes is necessary:

The following commands establish the basic setup shown:

(1) # tc qdisc replace dev eth0 root handle 1: htb default 30
(2) # tc class add dev eth0 parent 1: classid 1:1 htb rate 95mbit
(3) # alias tclass='tc class add dev eth0 parent 1:1'
(4) # tclass classid 1:10 htb rate 1mbit ceil 20mbit prio 1
(4) # tclass classid 1:20 htb rate 90mbit ceil 95mbit prio 2
(4) # tclass classid 1:30 htb rate 1mbit ceil 95mbit prio 3
(5) # tc qdisc add dev eth0 parent 1:10 fq_codel
(5) # tc qdisc add dev eth0 parent 1:20 fq_codel
(5) # tc qdisc add dev eth0 parent 1:30 fq_codel

A little explanation for the unfamiliar reader:

1. Replace the root qdisc of eth0 by an instance of HTB. Specifying the handle is necessary so it can be referenced in consecutive calls to tc. The default class for unclassified traffic is set to 30.
2. Create a single top-level class with handle 1:1 which limits the total bandwidth allowed to 95mbit/s. It is assumed that eth0 is a 100mbit/s link, staying a little below that helps to keep the main point of enqueueing in the qdisc layer instead of the interface hardware queue or at another bottleneck in the network.
3. Define an alias for the common part of the remaining three calls in order to improve readability. This means all remaining classes are attached to the common parent class from (2).
4. Create three child classes for different uses: Class 1:10 has highest priority but is tightly limited in bandwidth - fine for interactive connections. Class 1:20 has mid priority and high guaranteed bandwidth, for high priority bulk traffic. Finally, there's the default class 1:30 with lowest priority, low guaranteed bandwidth and the ability to use the full link in case it's unused otherwise. This should be fine for uninteresting traffic not explicitly taken care of.
5. Attach a leaf qdisc to each of the child classes created in (4). Since HTB by default attaches pfifo as leaf qdisc, this step is optional. Still, the fairness between different flows provided by the classless fq_codel is worth the effort.

More information about the qdiscs and fine-tuning parameters can be found in tc-htb(8) and tc-fq_codel(8).

Without any additional setup done, now all traffic leaving eth0 is shaped to 95mbit/s and directed through class 1:30. This can be verified by looking at the Sent field of the class statistics printed via tc -s class show dev eth0: Only the root class 1:1 and it's child 1:30 should show any traffic.