One of the most difficult aspects of working with the advanced routing features of linux is gaining an understanding the sequence of events as a packet traverses the kernel space. It is, in fact, the key knowledge needed to grasp the potential of advanced routing scenarios and to troubleshoot successfully when things don't go as planned.
If you are reading this for the first time, stop now and go visit and study the kernel packet traveling diagram and the kernel packet handling diagram now. These represent two different efforts to describe the order in which different networking subsystems inside the linux kernel have an opportunity to inspect, manipulate and redirect a packet. Understanding this sequence of events is key to harnessing the power of linux networking.
Now, let's examine some of the different commands you can use to manipulate packets at each of these stages. The list below describes the sequence of events for a packet bound for a non-local destination.
Packet Traversal; Non-Local Destination
All of the PREROUTING netfilter hooks are called here. This means that we get our first opportunity to inspect and drop a packet, we can perform DNAT on the packet to make sure that the destination IP is rewritten before we make a routing decision (at which time the destination address becomes very important). We can also set ToS or an fwmark on the packet at this time. If we want to use an IMQ device for ingress control, we can put our hooks here.
If we are using ipchains, the input chain is traversed.
Any traffic control on the real device on which the packet arrived is now performed.
The input routing stage is traversed by any packet entering the local machine. Here we concern ourselves only with packets which are routed through this machine to another destination Additionally, iproute2 NAT occurs here [36].
The packet enters the FORWARD netfilter hooks. Here, the packet can be mangled with ToS or fwmark. After the mangle chain is passed, the filter chain will be traversed. For kernel 2.4-based routing devices this will be the location for packet filtering rules. If we are using ipchains, the forward chain would be traversed here instead of the netfilter FORWARD hooks.
The output chain in an ipchains installation would be traversed here.
The POSTROUTING netfilter hooks are traversed. These include packet mangling, NAT and IMQ for egress.
Finally, the packet is transmitted via the outbound device per traffic control configuration on that outbound device.
The above describes the sequence of events for packets passing through the linux routing device. Let's look at a similar descriptions of the paths that packets bound for local destinations take through the kernel.
Packet Traversal; Local Destination
All of the PREROUTING netfilter hooks are called here. This means that we get our first opportunity to inspect and drop a packet, we can perform DNAT on the packet to make sure that the destination IP is rewritten before we make a routing decision (at which time the destination address becomes very important). We can also set ToS or an fwmark on the packet at this time. If we want to use an IMQ device for ingress control, we can put our hooks here.
If we are using ipchains, the input chain is traversed.
Any traffic control on the real device on which the packet arrived is now performed.
The input routing stage is traversed by any packet entering the local machine. Here we concern ourselves with packets bound for local destinations only.
The INPUT netfilter hooks are traversed. Commonly this is filtering for inbound connections, but can include packet mangling.
The local destination process receives the connection. If there is no open socket, an error is generated.
Naturally, packets need to go out from the machine as well, so let's look at the path for outbound packets which were locally generated.
Packet Traversal; Locally Generated
The process with the open socket sends data.
The routing decision is made. This is frequently called output routing because it is only for packets leaving the system. This routing code is (sometimes?) responsible for selecting the source IP of the outbound packet.
The netfilter OUTPUT hooks are traversed. The basic filter, nat, and mangle hooks are available. This is where SNAT can take place.
The output chain in an ipchains installation would be traversed here.
The POSTROUTING netfilter hooks are traversed. These include packet mangling, NAT and IMQ for egress.
Finally, the packet is transmitted via the outbound device per traffic control configuration on that outbound device.
[36] Leonardo calls this "dumb NAT" because the NAT performed by iproute2 at the routing stage is stateless.