4.8. Routing Tables

Linux kernel 2.2 and 2.4 support multiple routing tables [22]. Beyond the two commonly used routing tables (the local and main routing tables), the kernel supports up to 252 additional routing tables.

The multiple routing table system provides a flexible infrastructure on top of which to implement policy routing. By allowing multiple traditional routing tables (keyed primarily to destination address) to be combined with the routing policy database (RPDB) (keyed primarily to source address), the kernel supports a well-known and well-understood interface while simultaneously expanding and extending its routing capabilities. Each routing table still operates in the traditional and expected fashion. Linux simply allows you to choose from a number of routing tables, and to traverse routing tables in a user-definable sequence until a matching route is found.

Any given routing table can contain an arbitrary number of entries, each of which is keyed on the following characteristics (cf. Table 4.1, “Keys used for hash table lookups during route selection”)

For practical purposes, this means that (even) a single routing table can contain multiple routes to the same destination if the ToS differs on each route or if the route applies to a different interface [23].

Kernels supporting multiple routing tables refer to routing tables by unique integer slots between 0 and 255 [24]. The two routing tables normally employed are table 255, the local routing table, and table 254, the main routing table. For examples of using multiple routing tables, see Chapter 9, Advanced IP Management, in particular, Example 10.1, “Multiple Outbound Internet links, part I; ip route, Example 10.3, “Multiple Outbound Internet links, part III; ip rule and Example 10.4, “Multiple Internet links, inbound traffic; using iproute2 only ”. Also be sure to read Section 10.3, “Using the Routing Policy Database and Multiple Routing Tables” and Section 4.9, “Routing Policy Database (RPDB)”.

The ip route and ip rule commands have built in support for the special tables main and local. Any other routing tables can be referred to by number or an administratively maintained mapping file, /etc/iproute2/rt_tables.

The format of this file is extraordinarily simple. Each line represents one mapping of an arbitrary string to an integer. Comments are allowed.

Example 4.6. Typical content of /etc/iproute2/rt_tables

#
# reserved values
#
255     local      1
254     main       2
253     default    3
0       unspec     4
#
# local
#
1      inr.ruhep   5
      
1 The local table is a special routing table maintained by the kernel. Users can remove entries from the local routing table at their own risk. Users cannot add entries to the local routing table. The file /etc/iproute2/rt_tables need not exist, as the iproute2 tools have a hard-coded entry for the local table.
2 The main routing table is the table operated upon by route and, when not otherwise specified, by ip route. The file /etc/iproute2/rt_tables need not exist, as the iproute2 tools have a hard-coded entry for the main table.
3 The default routing table is another special routing table, but WHY is it special!?!
4 Operating on the unspec routing table appears to operate on all routing tables simultaneously. Is this true!? What does that imply?
5 This is an example indicating that table 1 is known by the name inr.ruhep. Any references to table inr.ruhep in an ip rule or ip route will substitue the value 1 for the word inr.ruhep.

The routing table manipulated by the conventional route command is the main routing table. Additionally, the use of both ip address and ifconfig will cause the kernel to alter the local routing table (and usually the main routing table). For further documentation on how to manipulate the other routing tables, see the command description of ip route.

4.8.1. Routing Table Entries (Routes)

Each routing table can contain an arbitrary number of route entries. Aside from the local routing table, which is maintained by the kernel, and the main routing table which is partially maintained by the kernel, all routing tables are controlled by the administrator or routing software. All routes on a machine can be changed or removed [25].

Each of the following route types is available for use with the ip route command. Each route type causes a particular sort of behaviour, which is identified in the textual description. Compare the route types described below with the rule types available for use in the RPDB.

unicast

A unicast route is the most common route in routing tables. This is a typical route to a destination network address, which describes the path to the destination. Even complex routes, such as nexthop routes are considered unicast routes. If no route type is specified on the command line, the route is assumed to be a unicast route.

Example 4.7. unicast route types

ip route add unicast 192.168.0.0/24 via 192.168.100.5
ip route add default via 193.7.255.1
ip route add unicast default via 206.59.29.193
ip route add 10.40.0.0/16 via 10.72.75.254
              
broadcast

This route type is used for link layer devices (such as Ethernet cards) which support the notion of a broadcast address. This route type is used only in the local routing table [26] and is typically handled by the kernel.

Example 4.8. broadcast route types

ip route add table local broadcast 10.10.20.255 dev eth0 proto kernel scope link src 10.10.20.67
ip route add table local broadcast 192.168.43.31 dev eth4 proto kernel scope link src 192.168.43.14
              
local

The kernel will add entries into the local routing table when IP addresses are added to an interface. This means that the IPs are locally hosted IPs [27].

Example 4.9. local route types

ip route add table local local 10.10.20.64 dev eth0 proto kernel scope host src 10.10.20.67
ip route add table local local 192.168.43.12 dev eth4 proto kernel scope host src 192.168.43.14
              
nat

This route entry is added by the kernel in the local routing table, when the user attempts to configure stateless NAT. See Section 5.3, “Stateless NAT with iproute2 for a fuller discussion of network address translation in general. [28].

Example 4.10. nat route types

ip route add nat 193.7.255.184 via 172.16.82.184
ip route add nat 10.40.0.0/16 via 172.40.0.0
              
unreachable

When a request for a routing decision returns a destination with an unreachable route type, an ICMP unreachable is generated and returned to the source address.

Example 4.11. unreachable route types

ip route add unreachable 172.16.82.184
ip route add unreachable 192.168.14.0/26
ip route add unreachable 209.10.26.51
              
prohibit

When a request for a routing decision returns a destination with a prohibit route type, the kernel generates an ICMP prohibited to return to the source address.

Example 4.12. prohibit route types

ip route add prohibit 10.21.82.157
ip route add prohibit 172.28.113.0/28
ip route add prohibit 209.10.26.51
              
blackhole

A packet matching a route with the route type blackhole is discarded. No ICMP is sent and no packet is forwarded.

Example 4.13. blackhole route types

ip route add blackhole default
ip route add blackhole 202.143.170.0/24
ip route add blackhole 64.65.64.0/18
              
throw

The throw route type is a convenient route type which causes a route lookup in a routing table to fail, returning the routing selection process to the RPDB. This is useful when there are additional routing tables. Note that there is an implicit throw if no default route exists in a routing table, so the route created by the first command in the example is superfluous, although legal.

Example 4.14. throw route types

ip route add throw default
ip route add throw 10.79.0.0/16
ip route add throw 172.16.0.0/12
              

The power of these route types when combined with the routing policy database can hardly be understated. All of these route types can be used without the RPDB, although the throw route doesn't make much sense outside of a multiple routing table installation.

4.8.2. The Local Routing Table

The local routing table is maintained by the kernel. Normally, the local routing table should not be manipulated, but it is available for viewing. In Example D.12, “Viewing the local routing table with ip route show table local, you'll see two of the common uses of the local routing table. The first common use is the specification of broadcast address, necessary only for link layers which support broadcast addressing. The second common type of entry in a local routing table is a route to a locally hosted IP.

The route types found in the local routing table are local, nat and broadcast. These route types are not relevant in other routing tables, and other route types cannot be used in the local routing table.

If the the machine has several IP addresses on one Ethernet interface, there will be a route to each locally hosted IP in the local routing table. This is a normal side effect of bringing up an IP address on an interface under linux. Maintenance of the broadcast and local routes in the local routing table can only be done by the kernel.

Example 4.15. Kernel maintenance of the local routing table

[root@real-server]# ip address show dev eth1
6: eth1: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100
   link/ether 00:80:c8:e8:1e:fc brd ff:ff:ff:ff:ff:ff
   inet 10.10.20.89/24 brd 10.10.20.255 scope global eth1
[root@real-server]# ip route show dev eth1
10.10.20.0/24  proto kernel  scope link  src 10.10.20.89
[root@real-server]# ip route show dev eth1 table local
broadcast 10.10.20.0  proto kernel  scope link  src 10.10.20.89 
broadcast 10.10.20.255  proto kernel  scope link  src 10.10.20.89
local 10.10.20.89  proto kernel  scope host  src 10.10.20.89
[root@real-server]# ip address add 192.168.254.254/24 brd + dev eth1
[root@real-server]# ip address show dev eth1
6: eth1: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100
   link/ether 00:80:c8:e8:1e:fc brd ff:ff:ff:ff:ff:ff
   inet 10.10.20.89/24 brd 10.10.20.255 scope global eth1
   inet 192.168.254.254/24 brd 192.168.254.255 scope global eth1
[root@real-server]# ip route show dev eth1
10.10.20.0/24  proto kernel  scope link  src 10.10.20.89 
192.168.254.0/24  proto kernel  scope link  src 192.168.254.254
[root@real-server]# ip route show dev eth1 table local
broadcast 10.10.20.0  proto kernel  scope link  src 10.10.20.89 
broadcast 192.168.254.0  proto kernel  scope link  src 192.168.254.254          
broadcast 10.10.20.255  proto kernel  scope link  src 10.10.20.89               
local 192.168.254.254  proto kernel  scope host  src 192.168.254.254            
local 10.10.20.89  proto kernel  scope host  src 10.10.20.89                    
broadcast 192.168.254.255  proto kernel  scope link  src 192.168.254.254
        

Note in Example 4.15, “Kernel maintenance of the local routing table”, that the kernel adds not only the route for the locally connected network in the main routing table, but also the three required special addresses in the local routing table. Any IP addresses which are locally hosted on the box will have local entries in the local table. The network address and broadcast address are both entered as broadcast type addresses on the interface to which they have been bound. Conceptually, there is significance to the distinction between a network and broadcast address, but practically, they are treated analogously, by other networking gear as well as the linux kernel.

There is one other type of route which commonly ends up in the local routing table. When using iproute2 NAT, there will be entries in the local routing table for each network address translation. Refer to Example D.21, “Creating a NAT route for a single IP with ip route add nat and Example D.22, “Creating a NAT route for an entire network with ip route add nat for example output.

4.8.3. The Main Routing Table

The main routing table is the routing table most people think of when considering a linux routing table. When no table is specified to an ip route command, the kernel assumes the main routing table. The route command only manipulates the main routing table.

Similarly to the local table, the main table is populated automatically by the kernel when new interfaces are brought up with IP addresses. Consult the main routing table before and after ip address add 192.168.254.254/24 brd + dev eth1 in Example 4.15, “Kernel maintenance of the local routing table” for a concrete example of this kernel behaviour. Also, visit this summary of side effects of interface definition and activation with ifconfig or ip address.



[22] The kernel must be compiled with the option CONFIG_IP_MULTIPLE_TABLES=y. This is common in vendor and stock kernels, both 2.2 and 2.4.

[23] If somebody has used scope or oif as additional keys in a routing table, and has an example, I'd love to see it, for possible inclusion in this documentation.

[24] Can anybody describe to me what is in table 0? It looks almost like an aggregation of the routing entries in routing tables 254 and 255.

[25] Once again, I recommend caution when altering the local routing table. Removing local route types from the local routing table can break networking in strange and wonderful ways.

[26] OK, I'm not absolutely sure you can't use the broadcast route in other routing tables, but I believe you can't. Testing forthcoming...

[27] Ibid. I'm not sure that local route types can be used in any routing table other than the local routing table. Testing forthcoming...

[28] Ibid. nat route types might be ineffectual outside the local routing table. Testing forthcoming...