Copyright © 2002, 2003, 2004, 2005, 2006, 2007 Martin A. Brown
"Mar 2007"
| Revision History | ||
|---|---|---|
| Revision 0.4.5 | 2007-03-31 | MAB |
| corrected DocBook build environment; new mail address | ||
| Revision 0.4.4 | 2003-04-26 | MAB |
| added index, began packet filtering chapter | ||
| Revision 0.4.3 | 2003-04-14 | MAB |
| ongoing editing, ARP/NAT fixes, routing content | ||
| Revision 0.4.2 | 2003-03-16 | MAB |
| ongoing editing; unreleased version | ||
| Revision 0.4.1 | 2003-02-19 | MAB |
| major routing revision; better use of callouts | ||
| Revision 0.4.0 | 2003-02-11 | MAB |
| major NAT revs; add inline scripts; outline FIB | ||
| Revision 0.3.9 | 2003-02-05 | MAB |
| fleshed out bonding; added bridging chapter | ||
| Revision 0.3.8 | 2003-02-03 | MAB |
| move to linux-ip.net; use TLDP XSL stylesheets | ||
| Revision 0.3.7 | 2003-02-02 | MAB |
| major editing on ARP; minor editing on routing | ||
| Revision 0.3.6 | 2003-01-30 | MAB |
| switch to XSLT processing; minor revs; CVS | ||
| Revision 0.3.5 | 2003-01-08 | MAB |
| ARP flux complete; ARP filtering touched | ||
| Revision 0.3.4 | 2003-01-06 | MAB |
| ARP complete; bridging added; ip neigh complete | ||
| Revision 0.3.3 | 2003-01-05 | MAB |
| split into 3 parts; ARP chapter begun | ||
| Revision 0.3.2 | 2002-12-29 | MAB |
| links updated; minor editing | ||
| Revision 0.3.1 | 2002-11-26 | MAB |
| edited: intro, snat, nat; split advanced in two | ||
| Revision 0.3.0 | 2002-11-14 | MAB |
| chapters finally have good HTML names | ||
| Revision 0.2.9 | 2002-11-11 | MAB |
| routing chapter heavily edited | ||
| Revision 0.2.8 | 2002-11-07 | MAB |
| basic chapter heavily edited | ||
| Revision 0.2.7 | 2002-11-04 | MAB |
| routing chapter finished; links rearranged | ||
| Revision 0.2.6 | 2002-10-29 | MAB |
| routing chapter continued | ||
| Revision 0.2.5 | 2002-10-28 | MAB |
| routing chapter partly complete | ||
| Revision 0.2.4 | 2002-10-08 | MAB |
| advanced routing additions and overview | ||
| Revision 0.2.3 | 2002-09-30 | MAB |
| minor editing; worked on tools/netstat; advanced routing | ||
| Revision 0.2.2 | 2002-09-24 | MAB |
| formalized revisioning; finished basic networking; started netstat | ||
| Revision 0.2.1 | 2002-09-21 | MAB |
| added network map to incomplete rough draft | ||
| Revision 0.2 | 2002-09-20 | MAB |
| incomplete rough draft released on LARTC list | ||
| Revision 0.1 | 2002-08-04 | MAB |
| rough draft begun | ||
Abstract
This guide provides an overview of many of the tools available for IP network administration of the linux operating system, kernels in the 2.2 and 2.4 series. It covers Ethernet, ARP, IP routing, NAT, and other topics central to the management of IP networks.
Table of Contents
List of Tables
List of Examples
conf/$DEV/arp_filternet/$DEV/hidden/etc/iproute2/rt_tableslocal routing tableREJECT
target, cf. Example D.17, “Adding a prohibit route with route
add”prohibit route with route
addfrom in a routing command with
route addsrc in a routing command with
route addTable of Contents
This guide is as an overview of the IP networking capabilities of linux kernels 2.2 and 2.4. The target audience is any beginning to advanced network administrator who wants practical examples and explanation of rumoured features of linux. As the Internet is lousy with documentation on the nooks and crannies of linux networking support, I have tried to provide links to existing documentation on IP networking with linux.
The documentation you'll find here covers kernels 2.2 and 2.4, although a good number of the examples and concepts may also apply to older kernels. In the event that I cover a feature that is only present or supported under a particular kernel, I'll identify which kernel supports that feature.
I assume a few things about the reader. First, the reader has a basic understanding (at least) of IP addressing and networking. If this is not the case, or the reader has some trouble following my networking examples, I have provided a section of links to IP layer tutorials and general introductory documentation in the appendix. Second, I assume the reader is comfortable with command line tools and the Linux, Unix, or BSD environments. Finally, I assume the reader has working network cards and a Linux OS. For assistance with Ethernet cards, the there exists a good Ethernet HOWTO.
The examples I give are intended as tutorial examples only. The user should understand and accept the ramifications of using these examples on his/her own machines. I recommend that before running any example on a production machine, the user test in a controlled environment. I accept no responsibility for damage, misconfiguration or loss of any kind as a result of referring to this documentation. Proceed with caution at your own risk.
This guide has been written primarily as a companion reference to IP networking on Ethernets. Although I do allude to other link layer types occasionally in this book, the focus has been IP as used in Ethernet. Ethernet is one of the most common networking devices supported under linux, and is practically ubiquitous.
This text was written in DocBook with vim. All formatting has been applied by xsltproc based on DocBook and LDP XSL stylesheets. Typeface formatting and display conventions are similar to most printed and electronically distributed technical documentation. A brief summary of these conventions follows below.
The interactive shell prompt will look like
[root@hostname]#
for the root user and
[user@hostname]$
for non-root users, although most of the operations we will be discussing will require root privileges.
Any commands to be entered by the user will always appear like
{ echo "Hi, I am exiting with a non-zero exit code."; exit 1 }
Output by any program will look something like this:
Hi, I am exiting with a non-zero exit code.
Where possible, an additional convention I have used is the suppression of all hostname lookup. DNS and other naming based schemes often confuse the novice and expert alike, particularly when the name resolver is slow or unreachable. Since the focus of this guide is IP layer networking, DNS names will be used only where absolutely unambiguous.
Perhaps this should be called things that are wrong with this document,
or things which should be improved. See the
src/ROADMAP for notes on what is likely to be
forthcoming in subsequent releases.
The internal document linking, while good, but could be better. Especially lame is the lack of an index. External links should be used more commonly where appropriate instead of sending users to the links page.
If you are looking for LARTC topics, you may find some LAR topics here, but you should try the LARTC page itself if you have questions that are more TC than LAR. Consult Appendix I, Links to other Resources for further references to available documentation.
There are many tools available under linux which are also available under other unix-like operating systems, but there are additional tools and specific tools which are available only to users of linux. This guide represents an effort to identify some of these tools. The most concrete example of the difference between linux only tools and generally available unix-like tools is the difference between the traditional ifconfig and route commands, available under most variants of unix, and the iproute2 command suite, written specificially for linux.
Because this guide concerns itself with the features, strengths, and peculiarities of IP networking with linux, the iproute2 command suite assumes a prominent role. The iproute2 tools expose the strength, flexibility and potential of the linux networking stack.
Many of the tools introduced and concepts introduced are also detailed in other HOWTOs and guides available at The Linux Documentation Project in addition to many other places on the Internet and in printed books.
As with many human endeavours, this work is made possible by the efforts of others. For me, this effort represents almost four years of learning and network administration. The knowledge collected here is in large measure a repackaging of disparate resources and my own experiences over time. Without the greater linux community, I would not be able to provide this resource.
I would like to take this opportunity to make a plug for my employer, SecurePipe, Inc. which has provided me stable and challenging employment for these (almost) four years. SecurePipe is a managed security services provider specializing in managed firewall, VPN, and IDS services to small and medium sized companies. They offer me the opportunity to hone my networking skills and explore areas of linux networking unknown to me. Thanks also to SecurePipe, Inc. for hosting this cost-free on their servers.
Over the course of the project, many people have contributed suggestions,
modifications, corrections and additions. I'll acknowledge them briefly
here. For full acknowledgements, see
src/ACKNOWLEDGEMENTS in the DocBook source tree.
Russ Herrold, 2002-09-22
Yann Hirou, 2002-09-26
Julian Anastasov, 2002-10-29
Bert Hubert, 2002-11-14
Tony Kapela, 2002-11-30
George Georgalis, 2003-01-11
Alex Russell, 2003-02-02
giovanni, 2003-02-06
Gilles Douillet, 2003-02-28
Please feel free to point out any irregularities, factual errors,
typographical errors, or logical gaps in this
documentation. If you have rants or raves about this documentation,
please mail me directly at <martin@linux-ip.net>.
Now, let's begin! Let me welcome you to the pleasure and reliability of IP networking with linux.
Table of Contents
Table of Contents
Internet Protocol (IP) networking is now among the most common networking technologies in use today. The IP stack under linux is mature, robust and reliable. This chapter covers the basics of configuring a linux machine or multiple linux machines to join an IP network.
This chapter covers a quick overview of the locations of the networking control files on different distributions of linux. The remainder of the chapter is devoted to outlining the basics of IP networking with linux.
These basics are written in a more tutorial style than the remainder of the first part of the book. Reading and understanding IP addressing and routing information is a key skill to master when beginning with linux. Naturally, the next step is to alter the IP configuration of a machine. This chapter will introduce these two key skills in a tutorial style. Subsequent chapters will engage specific subtopics of linux networking in a more thorough and less tutorial manner.
Different linux distribution vendors put their networking configuration files in different places in the filesystem. Here is a brief summary of the locations of the IP networking configuration information under a few common linux distributions along with links to further documentation.
Location of networking configuration files
RedHat (and Mandrake)
Interface definitions
/etc/sysconfig/network-scripts/ifcfg-*
Hostname and default gateway definition
/etc/sysconfig/network
Definition of static routes
/etc/sysconfig/static-routes
SuSe (version >= 8.0)
Interface definitions
/etc/sysconfig/network/ifcfg-*
Static route definition
/etc/sysconfig/network/routes
Interface specific static route definition
/etc/sysconfig/network/ifroute-*
SuSe (version <= 8.0)
Interface and route definitions
/etc/rc.config
Debian
Interface and route definitions
/etc/network/interfaces
Gentoo
Interface and route definitions
/etc/conf.d/net
Slackware
Interface and route definitions
/etc/rc.d/rc.inet1
The format of the networking configuration files differs significantly from distribution to distribution, yet the tools used by these scripts are the same. This documentation will focus on these tools and how they instruct the kernel to alter interface and route information. Consult the distribution's documentation for questions of file format and order of operation.
For the remainder of this document, many examples refer to machines in a hypothetical network. Refer to the example network description for the network map and addressing scheme.
Assuming an already configured machine named tristan, let's
look at the IP addressing and routing
table. Next we'll examine how the machine
communicates with computers (hosts) on the locally reachable network. We'll
then send packets through our
default gateway to other networks. After learning what a default
route is, we'll look at a static
route.
One of the first things to learn about a machine attached to an IP
network is its IP address. We'll begin by looking at
a machine named tristan on the main desktop network (192.168.99.0/24).
The machine tristan
is alive on IP 192.168.99.35 and
has been properly configured by the system administrator.
By examining the
route
and ifconfig
output we can learn a good deal about the network to which
tristan is connected
[1].
Example 1.1. Sample ifconfig output
|
For the moment, ignore the loopback interface (lo) and concentrate on the Ethernet interface. Examine the output of the ifconfig command. We can learn a great deal about the IP network to which we are connected simply by reading the ifconfig output. For a thorough discussion of ifconfig, see Section C.1, “ifconfig”.
The IP address active on tristan is 192.168.99.35. This means that
any IP packets created by tristan will have a
source address of 192.168.99.35. Similarly any packet received by
tristan will have the destination address of 192.168.99.35.
When creating an outbound packet tristan will set the destination
address to the server's IP. This gives the remote host and the
networking devices in between these hosts enough information to
carry packets between the two devices.
Because tristan will
advertise that it accepts packets with a destination address of
192.168.99.35, any frames (packets) appearing on the Ethernet
bound for 192.168.99.35 will reach tristan. The process of
communicating the ownership of an IP address is called ARP. Read
Section 2.1.1, “Overview of Address Resolution Protocol” for a complete discussion of
this process.
This is fundamental to IP networking. It is fundamental that a host be able to generate and receive packets on an IP address assigned to it. This IP address is a unique identifier for the machine on the network to which it is connected.
Common traffic to and from machines today is unicast IP traffic.
Unicast traffic is essentially a conversation between two hosts.
Though there may be routers between them, the two hosts are carrying
on a private conversation. Examples of common unicast traffic
are protocols such as HTTP (web), SMTP (sending mail), POP3 (fetching
mail), IRC (chat), SSH (secure shell), and LDAP (directory access).
To participate in any of these kinds of traffic,
tristan will send and receive packets on 192.168.99.35.
In contrast to unicast traffic, there is another common IP networking technique called broadcasting. Broadcast traffic is a way of addressing all hosts in a given network range with a single destination IP address. To continue the analogy of the unicast conversation, a broadcast is more like shouting in a room. Occasionally, network administrators will refer to broadcast techniques and broadcasting as "chatty network traffic".
Broadcast techniques are used at the Ethernet layer and the IP layer, so the cautious person talks about Ethernet broadcasts or IP broadcast. Refer to Section 2.1.1, “Overview of Address Resolution Protocol”, for more information on a common use of broadcast Ethernet frames.
IP Broadcast techniques can be used to share information with all partners on a network or to discover characteristics of other members of a network. SMB (Server Message Block) as implemented by Microsoft products and the samba package makes extensive use of broadcasting techniques for discovery and information sharing. Dynamic Host Configuration Protocol (DHCP) also makes use of broadcasting techniques to manage IP addressing.
The IP broadcast address is, usually, correctly derived from the IP address and network mask although it can be easily be set explicitly to a different address. Because the broadcast address is used for autodiscovery (e.g, SMB under some protocols, an incorrect broadcast address can inhibit a machine's ability to participate in networked communication [2].
The netmask on the interface should match the netmask in the routing table for the locally connected network. Typically, the route and the IP interface definition are calculated from the same configuration data so they should match perfectly.
If you are at all confused about how to address a network or how to read either the traditional notation or the CIDR notation for network addressing, see one of the CIDR/netmask references in Section I.1.3, “General IP Networking Resources”.
We can see from the output above that the IP address 192.168.99.35
falls inside the address space 192.168.99.0/24. We also note that
the machine tristan
will route packets bound for 192.168.99.0/24 directly onto the
Ethernet attached to eth0. This line in the routing table
identifies a network available on the Ethernet attached to eth0
("Iface") by its network address ("Destination") and size ("Genmask").
|
Every host on the 192.168.99.0/24 network should share the network address and netmask specified above. No two hosts should share the same IP address.
Currently, there are two hosts connected to the example desktop network.
Both tristan and masq-gw are connected to 192.168.99.0/24. Thus,
192.168.99.254 (masq-gw) should be reachable from tristan.
Success of this test provides evidence that tristan is
configured properly. N.B., Assume that the network
administrator has properly configured masq-gw. Since the
default gateway in any
network is an important host, testing reachability of the default
gateway also has a value in determining the proper operation of the
local network.
The ping tool, designed to take advantage of Internet Control Message Protocol (ICMP), can be used to test reachability of IP addresses. For a command summary and examples of the use of ping, see Section G.1, “ping”.
Example 1.2. Testing reachability of a locally connected host with ping
|
In Section 1.2.1, “Sending Packets to the Local Network”, we verified that hosts connected to the same local network can reach each other and, importantly, the default gateway. Now, let's see what happens to packets which have a destination address outside the locally connected network.
Assuming that the network administrator allows ping packets
from the desktop network into the public network,
ping can be invoked with the
record route option to show the path the packet travels from
tristan to wan-gw and back.
Example 1.3. Testing reachability of non-local hosts
By testing reachability of the local network 192.168.99.0/24 and an IP address outside our local network, we have verified the basic elements of IP connectivity.
To summarize this section, we have:
identified the IP address, network address and netmask in use
on tristan using the tools ifconfig and
route
verified that tristan can reach its default gateway
tested that packets bound for destinations outside our local network reach the intended destination and return
Static routes instruct the kernel to route packets
for a known destination host or network to a router or
gateway different from the default gateway.
In the example network, the desktop machine tristan would need
a static route to reach hosts in the 192.168.98.0/24 network.
Note that the branch office network is reachable over an ISDN line.
The ISDN router's IP in tristan's network is 192.168.99.1. This
means that there are two gateways in the example desktop network,
one connected to a small branch office network, and the other
connected to the Internet.
Without a static route to the branch office network, tristan would
use masq-gw as the gateway, which is not the most efficient path for
packets bound for morgan. Let's examine why a static route would
be better here.
If tristan generates a packet bound for morgan and
sends the packet to the default gateway, masq-gw will forward the
packet to isdn-router as well as generate an ICMP redirect message
to tristan. This ICMP redirect message tells tristan to send
future packets with a destination address of 192.168.98.82 (morgan)
directly to isdn-router. For a fuller discussion of ICMP redirect,
see
Section 4.10.2, “ICMP Redirects and Routing”.
The absence of a static route has caused two extra packets to be
generated on the Ethernet for no benefit. Not only that, but
tristan will eventually expire the temporary route entry
[3]
for 192.168.98.82, which means that subsequent packets bound for
morgan will repeat this process
[4].
To solve this problem, add a static route to tristan's routing
table. Below is a modified routing table (see
Section 1.3, “Changing IP Addresses and Routes” to learn how to change the routing
table).
Example 1.4. Sample routing table with a static route
|
According to this routing table, any packets with a destination address in the 192.168.98.0/24 network will be routed to the gateway 192.168.99.1 instead of the default gateway. This will prevent unnecessary ICMP redirect messages.
These are the basic tools for inspecting the IP address and the routes on a linux machine. Understanding the output of these tools will help you understand how machines fit into simple networks, and will be a base on which you can build an understanding of more complex networks.
[1] For BSD and UNIX users, the idiom netstat -rn may be more familiar than the common route -n on a linux machine. Both of these commands provide the same basic information although the formatting is a bit different. For a fuller discussion of these, see either Section G.4, “netstat” or Section D.1, “route”. For access to all of the routing features of the linux kernel, use ip route instead.
[2] An incorrect broadcast address often highlights a mismatch of the configured IP address and netmask on an interface. If in doubt, be sure to use an IP calculator to set the correct netmask and broadcast addresses.
[3] If the machine is a linux machine, then the temporary route entry is stored in the routing cache. Consult Section 4.7, “Routing Cache” for more information on the routing cache.
[4] It is quite reasonable to ignore ICMP redirect messages from unknown hosts on the Internet, but ICMP redirect messages on a LAN indicate that a host has mismatched netmasks or missing static routes.
This section introduces changing the IP address on an interface, changing the default gateway, and adding and removing a static route. With the knowledge of ifconfig and route output it's a small step to learn how to change IP configuration with these same tools.
For a practical example, let's say that the branch office server,
morgan, needs to visit the main office for some hardware maintenance.
Since the services on the machine are not in use, it's a convenient
time to fetch some software updates, after configuring the machine to
join the LAN.
Once the machine is booted and connected to the Ethernet, it's ready for IP reconfiguration. In order to join an IP network, the following information is required. Refer to the network map and appendix to gather the required information below.
Example 1.5. ifconfig and route output before the change
|
The process of readdressing for the new network involves three steps.
It is clear in
Example 1.5, “ifconfig and route
output before the change”, that morgan is configured
for a different network than the main office desktop network.
First, the
active interface must be
brought down, then a
new address must be configured
on the interface and brought up, and finally
a new default route must be
added. If the networking configuration is correct and the
process is successful, the machine should be able to connect to local
and non-local destinations.
This is a fast way to stop networking on a single-homed machine such as a server or workstation. On multi-homed hosts, other interfaces on the machine would be unaffected by this command. This method of bringing down an interface has some serious side effects, which should be understood. Here is a summary of the side effects of bringing down an interface.
Side effects of bringing down an interface with ifconfig
all IP addresses on the specified interface are deactivated and removed
any connections established to or from IPs on the specified interface are broken [7]
all routes to any destinations through the specified interface are removed from the routing tables
the link layer device is deactivated
The next step, bringing up the interface, requires the new networking configuration information. It's a good habit to check the interface after configuration to verify settings.
Example 1.7. Bringing up an Ethernet interface with ifconfig
|
The second call to ifconfig allows verification of the IP addressing information. The currently configured IP address on eth0 is 192.168.99.14. Bringing up an interface also has a small set of side effects.
Side effects of bringing up an interface
the link layer device is activated
the requested IP address is assigned to the specified interface
all local, network, and broadcast routes implied by the IP configuration are added to the routing tables
Use ping to verify the reachability of other locally connected hosts or skip directly to setting the default gateway.
It should come as no surprise to a close reader
(hint),
that the default route was removed at the execution of
ifconfig eth0 down. The crucial final step is
configuring the default route.
Example 1.8. Adding a default route with route
|
The routing table on morgan should look exactly like the initial
routing table on tristan. Compare the routing tables in