Cerberus: A Bridging Firewall with Linux

Overview
Introduction
Recompiling the kernel
Configuring Network Interface Cards
Configuring TCP/IP
brcfg
Filtering
References

Overview

This article is outdated. It was written before bridging+firewall functionality was fully supported in the Linux kernel. It is suggested that you use a recent kernel and follow the Ethernet Bridge + netfilter Howto.

This article describes how to build a bridging, packet-filtering firewall using Linux. The version of Linux used was a 2.2.5 kernel from a RedHat 6.0 distribution. It should also work with 2.2.12 from RedHat 6.1. It will not work with 2.2.14 and beyond because there were major architectual changes to the bridging code in 2.2.14.

The motivation for writing this paper was the frustration I experienced while getting bridging to work. My attempts at getting the bridging code to function were hampered with outdated documentation and tools. The Bridge mini-HOWTO is out of date, and there aren't that many resources available on Linux bridging. I hope no one will have to search through 500 articles on DejaNews after reading this paper. This paper is not an attempt at rewriting the HOWTOs, but to present their content in a digested form pertinent to bridging.

Introduction

Why another firewall for Linux? The answer is there are some cases when you want to bridge instead of route. There are advantages to bridging:
Transparency
Bridging is transparent, so you do not have to reconfigure the hosts on your existing network with new default routes, etc. For large networks, this can be a major win. Just slip in the bridge between your two network segments and off you go.
Empowerment
Bridging gives you control if you are an end user. If you are in an environment where you do not control your organization's firewall and don't trust the people who do, you can setup a bridging firewall so that your machines are protected. This is good news if you don't know what's at the other end of that 10BaseT cable in your office.
This document describes how to patch the Linux kernel to filter bridged packets through the firewall code. This solution is not for everyone, however. There are some drawbacks to this approach:
Experimental
Bridging is listed as EXPERIMENTAL in the kernel compile options. I agree. The Linux bridging code has a legacy of being unsupported (although that is changing; see this article in the linux-kernel list). My personal experiences have led me to believe that you can get bridging to work if you have the right network interface cards.
Packet filtering only
Packet filtering does not perform stateful packet inspection like modern kernels with iptables do. It will only block packets based on static ipchains rules.
Only filters IP
The packet filter hook in the bridging code can filter IP packets only. That's it. You can, however, block non-IP protocols completely. If all you want to filter is IP, then this is not a problem.
Can DENY packets, but not REJECT them
The patch to the bridging code will not send ICMP error messages in response to packets you want to reject. It can only discard them silently.
Ethernet only
Sorry token ring fans. The bridging code only works with Ethernet. You'll have to build a routing firewall if you have token ring.
The firewalling capabilities of Linux are quite extensive. If you are routing between different subnets, have your own virtual private network, or are doing tricky IP masquerading, a routing solution is the way to go.

Recompiling the kernel

You will need to install the kernel source and headers manually (or specify "kernel development" from the RedHat package list at install time).

If you want to include firewall support in the bridging code, patch br.c with this patch. The patch filters all IP packets through the kernel's firewall code. The original idea came from JJ Reynolds.

cd /usr/src/linux/net/bridge
cp br.c br.c.orig
patch br.c br.c.patch
Now you need to recompile the kernel with firewalling support built in.
cd /usr/src/linux
make menuconfig  (or xconfig or whatever)
Here are the minimal options you want to select under Networking options:
Network firewalls=y
IP firewalling=y
IP firewall packet netlink device=y
IP optimize as router not host=y
Bridging=y
Next, recompile the kernel. Here are the steps I use to do this:
make dep
make clean
make bzImage
make modules
make modules_install
The next step is to install the new kernel so that it loads the next time you reboot. I like to manually configure /etc/lilo.conf myself to make sure everything is the way I want it:
vi /etc/lilo.conf   (edit whatever you need to)
cp /usr/src/linux/arch/i386/boot/bzImage /boot
cp /usr/src/linux/System.map /boot
/sbin/lilo

Configuring Network Interface Cards

The Ethernet HOWTO is the definitive guide to this subject. See the section entitled Using More than one Ethernet Card per Machine for details.

You need to make sure your NICs use different interrupts and i/o ports. For ISA cards you will probably do this with jumpers on the card. For PCI cards (or ISA cards without jumpers), I configure the interrupts using my BIOS setup program since I don't like messing with plug-and-play tools. Most BIOSes have a setting called PnP OS. If you set this to No or False, it means that the OS will not be able to override the irqs that the BIOS reports. I like this setting. I can manually configure my BIOS to give my PCI cards unique interrupts and forget about messing with Linux plug-and-play tools or passing special boot parameters to LILO.

Warning: I could not get bridging to work with the SMC 1211TX card. I spent countless hours of my life trying. It works fine by itself, but not with the bridging code. I had lots of weird errors like packets being dropped on the wire if I did a ping of size 1469 (where fragmentation begins) from an NT box to a Linux box on the other side of the bridge. I went out and bought a 3Com 3C905B-TX and the problems went away. I recommend 3Com cards. I have successfully used 3C900B-TPO, 3C509B, and 3C905B-TX cards. I have also built a bridge out of a 486 with two $13USD ne2000 clone cards! It will work if you have the right Ethernet card. Get yourself some nice 3Com Ethernet cards and save yourself a lot of headaches.

Update: this problem was caused by a bug in br.c. See this article on the Linux kernel mailing list for more information. This bug was fixed in the 2.2.12 kernel.

If you have two PCI NICs, your /etc/conf.modules file will list each card like this:

alias eth0 3c59x
alias eth1 3c59x
Don't give i/o or irq information for PCI cards, even if the two PCI NICs are identical models. They will be found automatically by the driver. If you have two identical ISA NICs, you will have to specify the i/o ports for each interface like this:
alias eth0 ne
alias eth1 ne
options ne io=0x300,0x280

Configuring TCP/IP

Now you need to decide whether the bridge will be transparent or not. If it is transparent, there will be no IP interfaces visible to other machines. The advantage is that no one can break into the bridge (other than sending malformed packets to it) since there is no IP address to connect to. This has no impact on the bridging code. In this mode, your netstat -nr output will look like this:
# netstat -nr
Kernel IP routing table
Destination Gateway Genmask         Flags MSS Window  irtt Iface
127.0.0.2   0.0.0.0 255.255.255.255 UH      0 0          0 eth0
127.0.0.3   0.0.0.0 255.255.255.255 UH      0 0          0 eth1
127.0.0.0   0.0.0.0 255.0.0.0       U       0 0          0 lo
127.0.0.0   0.0.0.0 255.0.0.0       U       0 0          0 eth0
127.0.0.0   0.0.0.0 255.0.0.0       U       0 0          0 eth1
My /etc/sysconfig/network looks like:
NETWORK=yes
FORWARD_IPV4=true
HOSTNAME=cerberus.mydomain.com
I removed the GATEWAY and GATEWAYDEV lines since there are no default routes from this box. No TCP/IP packets originate from this box (only 802.1D bridging packets). It's important to have the FORWARD_IPV4=true line present.

My ifcfg scripts in /etc/sysconfig/network-scripts look like this:

# cat ifcfg-eth0
DEVICE=eth0
IPADDR=127.0.0.2
NETMASK=255.0.0.0
NETWORK=127.0.0.0
BROADCAST=127.255.255.255
ONBOOT=yes

# cat ifcfg-eth1
DEVICE=eth1
IPADDR=127.0.0.3
NETMASK=255.0.0.0
NETWORK=127.0.0.0
BROADCAST=127.255.255.255
ONBOOT=yes

The disadvantage to this approach is that you won't be able to ping machines from the bridge (or connect to any other service for that matter). You also won't be able to telnet to the machine; you will only be able to login on the console. The machine must be a dedicated bridge and nothing else.

The other approach is to give each interface a routeable IP address. The advantages here are that you can use the machine for other things and don't have to login to the console all the time, but you have to lock it down for it to be secure. Regardless of which method you choose, nothing needs to change on the other hosts on your network. If you choose to give the bridge routeable IP addresses, do not specify a default route on your hosts that corresponds to the bridge! A bridge is transparent and does not route packets. Also note that you will not be able to use tcpdump on the bridge because the bridging code runs in promiscuous mode and intercepts the packets before tcpdump does.

brcfg

brcfg is the program used to configure bridging behavior. It uses ioctl() calls to tell the bridging code in the kernel how to behave. This program is the source of much grief and frustration. There are two known versions floating around. One (brcfg from the 1.x days) is ancient and doesn't compile under 2.2 without some tweaking. The other (bridgex from Debian) has bugs. Neither program has been updated for the 2.2 world.

I am offering my own version of brcfg to the world. It is a modified Debian bridgex program. I fixed some bugs and added some new functionality.

You should be able to type "make" and get brcfg. The compiler will warn you about struct sk_buff, but this is okay since brcfg doesn't use skbuffs. Including kernel #includes and user-level #includes in the same file makes things messy, but trust me and ignore the compiler warning. I put my brcfg in /usr/local/bin. You must be root to use it. Type brcfg -help for usage. Now you are ready to start bridging.

  1. Put your interfaces into promiscuous mode:
    ifconfig eth0 promisc
    ifconfig eth1 promisc
  2. Start up bridging with brcfg start.
  3. Do brcfg to see the status. You should see some something like:
    bridging is ENABLED 	debugging is DISABLED	prot-stats are DISABLED
    spanning is ENABLED
    bridge id		0x0080 00:50:04:76:59:7b
    designated root		0x0080 00:50:04:76:59:7b
    bridge max age		20	max age			20
    bridge hello time	2	hello time		2
    bridge forward delay	15	forward delay		15
    root path cost		0	root port		0
    flags			TOPOLOGY_CHANGE TOPOLOGY_CHANGE_DETECTED
    --- port stats ---
    port 1	port id 0x8001	port state	LISTENING (0x1)
    designated root		0x0080 00:50:04:76:59:7b
    designated bridge	0x0080 00:50:04:76:59:7b
    path cost		0	designated cost		0
    designated port		32769	flags			NONE
    port 2	port id 0x8002	port state	LISTENING (0x1)
    designated root		0x0080 00:50:04:76:59:7b
    designated bridge	0x0080 00:50:04:76:59:7b
    path cost		0	designated cost		0
    designated port		32770	flags			NONE
    Policy                  Accept all protocols
    exempt protocols        0 
    
    The ports are in the LISTENING state. What is happening is the bridge is sending out IEEE 802.1D packets listening for another bridge to talk to. If it finds one, it will negotiate to find out who has the best spanning tree (path) for bridging. You should see the port states change from LISTENING to LEARNING to FORWARDING. Bridging should commence after about 30 seconds.
  4. Unfortunately, your Ethernet card may not work correctly with bridging. For some cards (like the SMC 1211TX), the Linux box will just sit there trying to negotiate with the non-existent bridges. If your brcfg output looks like this:
    bridge max age		20	max age			5120
    bridge hello time	2	hello time		512
    bridge forward delay	15	forward delay		3840
    
    then your Linux box will wait almost 3 hours (the above numbers are in seconds) until it will begin forwarding. If this is the case, you are doomed. Even if you force the bridge to forward packets using ioctl() hacks, you will experience weird errors like timeouts and dropped packets. Get a new Ethernet card. If all is well, the bridge should go into FORWARDING mode after half a minute or so.
  5. You can turn off the spanning tree algorithm if there are no other bridges on your network with brcfg span. Do another brcfg command and you should see that spanning is DISABLED and that the ports are now in the FORWARDING state. Now no IEEE 802.1D packets will be sent out from the bridge. It is important to note that you should not disable the spanning tree algorithm if there are other bridges on your network! If you do, you could mess up your network because packets can get caught in forwarding loops. I recommend you leave spanning on unless you are absolutely sure there are no other bridges on your network.
  6. Machines on each of your segments should now be able to ping each other. I like to start a continuous ping (default behavior on Linux, use the -t option for Windows machines) on one machine on each segment and make sure they can see each other. You should be able to switch the cables on the ports and have pings resume after about 10 seconds of delay where they fail. After that time, the machines will send a broadcast arp and the bridge will begin forwarding again.
  7. The last step is to put a few lines in your startup scripts so that everything starts up automatically when you boot. I put the following lines in /etc/rc.d/rc.local :
    ifconfig eth0 promisc
    ifconfig eth1 promisc
    /usr/local/bin/brcfg start
    
I would like to hear success stories from people. Please include the make, model, and number of Ethernet cards you used.

Filtering

Now that bridging works, it's time to worry about security. You can use brcfg to filter at the MAC level and ipchains to filter at the IP level (if you patched the kernel in the above step). The following contrived example allows WWW browsing only (not even DNS).
ifconfig eth0 promisc
ifconfig eth1 promisc
brcfg policy reject
brcfg exempt IP ARP
ipchains -P forward DENY
ipchains -A forward -p tcp --dport www -j ACCEPT
ipchains -A forward -p tcp --sport www ! --syn -j ACCEPT
brcfg start
The bridging policy is to reject all protocols except IP and ARP. You'll need ARP or bridging won't work. The forwarding policy is to allow web access in either direction. The bridge uses the forward chain to do its filtering. Note that if you are specifying the interface using ipchain's -i option, the packets traversing the forward chain use the interface they will go out on. See the IPCHAINS HOWTO for more information.

You may see packets traversing the chain even though the source and destination are on the same side of the bridge and there is no need for bridging. This is expected behavior. If the bridge has never heard of a particular Ethernet address, it will flood the packet on all of its interfaces so that the unknown machine can receive it. You will see a lot of this behavior when the bridge machine is rebooted.

References

You can find the HOWTO Index on tldp.org or locally on your machine in /usr/doc/HOWTO if you installed them from your Linux CDROM.
Bridge mini-HOWTO
This document is out of date and was the impetus behind this paper.
Bridge+Firewall mini-HOWTO
This document describes how to build a routing firewall.
Ethernet Bridge + netfilter Howto
Use this HOWTO instead of this article.
Charles Spurgeon's website
Charles Spurgeon talks about bridging in his books Practical Networking with Ethernet and Ethernet: The Definitive Guide
Ethernet HOWTO
A very well written document on Linux Ethernet.
Firewall HOWTO
This is really more geared to proxy firewalls.
IEEE 802.1D specs
This is a huge .pdf file. Not for the fainthearted. A real snooze-fest.
IPCHAINS HOWTO
An awesome HOWTO.
kfirewall 3.0 on freshmeat.net
An X gui for ipchains. I have never used it.