Quick update: I built a Debian Squeeze box recently and the same steps worked for me, initially. But problems developed afterwards. Specifically, net-agent (snmp) would take up more and more CPU over time. A "process leak", if you will. After months of monitoring and diagnostics we found a solution which involves a different NIC configuration.

So you need a fat network connection on your Debian 5 server, but a 10 GoE infrastructure is not in your budget? No fear! Link Aggregation Control Protocol is here to save the day!

This was both the problem I had, and the solution I learned to implement thanks to a question posted on serverfault.com (Multiplexed 1 Gbps Ethernet?) and some hours of research & experimentation.

Here's what I did:

• Purchase a NIC capable of LACP with solid Linux driver support. I went with an Intel PRO/1000 PT Quad Port Server Adapter.
• Purchase a Switch capable of LACP. It should be more than capable of handeling the bandwidth. We opted for an HP ProCurve 2510G-24.
• Install ifenslave-2.6: apt-get install ifenslave-2.6
• Purchase some network cables. Cat 6 if you can, Cat 5e if not.
• Edit /etc/modules and add bonding mode=4 miimon=100 max_bonds=2. This will load the module at boot time in the future. The value of max_bonds is the number of bonding devices your system will have. The default is 1.
• Load the module, so we can proceed. modprobe bonding. There should be no errors. Confirm it is loaded with modprobe -l | grep bond
• Edit /etc/network/interfaces to look something like this:
  

  # The loopback network interface
  auto lo
  iface lo inet loopback
  
  auto  bond0
  iface bond0 inet static
      address     192.168.1.131
      gateway     192.168.1.1
      broadcast   192.168.1.255
      netmask     255.255.255.0
      up   /sbin/ifenslave    bond0 eth0 eth1 eth2 eth3
      down /sbin/ifenslave -d bond0 eth0 eth1 eth2 eth3
  
  #similar configuration for bond1
  

• Add to /etc/modprobe.d/arch/X86_64.conf something like this for a 64-bit installation:
  

  
  alias bond0 bonding
  alias bond1 bonding
  

• Plug in the network cables
  
  
  
• Restart the network: /etc/init.d/network restart
• Configure the switch to use the four ports as a single trunk. Here's how for an HP ProCurve:

• Confirm that there are no errors
  1. The NIC and switch LEDs should be active
  2. dmesg will show what the kernal thinks of your new configuration
  3. ifconfig should show your bond0 interface up
  4. The interface should be pingable both internally and from other computer

You should be good to go at this point!

While there is some information out there on the Intertubes covering how to implement 80-->8080 port on a Linux box, they mainly pertain to single instance servers. The below iptables-restore input snippet works just great for such a setup:

*nat
#valid for tomcat listen to everywhere
-A PREROUTING -p tcp --dport 80 -j REDIRECT --to-ports 8080

#-A PREROUTING -j LOG --log-prefix "iptables routing: "  --log-level 7

COMMIT

But this configuration fails if your tomcat is configured to listen to a specific IP, as in when sharing a single server with multiple web servers. (Note the proxyPort attribute. This is important too.)

    <Connector port="8080" 
               proxyPort="80" 
               address="2.3.4.5"
               protocol="HTTP/1.1"
             >

No fear! A slightly more verbose iptables configuration file will do the trick!

cat /etc/iptables.up.rules
*nat
-A PREROUTING --dst 2.3.4.5 -p tcp --dport 80 -j DNAT --to-destination 2.3.4.5:8080
-A PREROUTING --dst 2.3.4.6 -p tcp --dport 80 -j DNAT --to-destination 2.3.4.6:8080
-A PREROUTING --dst 2.3.4.7 -p tcp --dport 80 -j DNAT --to-destination 2.3.4.7:8080

#-A PREROUTING -j LOG --log-prefix "iptables routing: "  --log-level 7

COMMIT

Presto! I worked this out with a bit of experimenting and a blog post I found: "Non-root Tomcat on Port 80 on a VPS without Apache"

Over the past few weeks I've been building up a new data center for my employer, xtendx AG. One of the core tasks has been to design, assemble and install a new storage system. To that end, I've put together the below NFSv4 Network Attached Storage (NAS) system.

• 3U 16-bay Chassis: SuperMicro SuperChassis 836A-R1200B
• LGA1156 Mother Board: Intel DP55KG w/ P55 Express chip set
• 2.67 MHz CPU: Intel Core i5 750
• 8GB DDR3 1366MHz RAM
• 3x 250 GB Hard Drives for OS: WD RE3 250GB WD2502ABYS
• 11x 2TB Hard Drives for Data: WD RE4 2TB WD2003FYYS
• RAID controller: LSI 3ware 9650SE-ML16
• Quad Port server NIC: Intel PRO/1000 PT Quad Port Server Adapter
• Operating System: Ubuntu 10.04 LTS

(larger picture)

Both the data and system volumes are in a RAID 10 configuration, with a hot spare for the OS already present. I'll be adding one or two hot spare 2TB drives in a few weeks.

Ubuntu is installed and mostly configured. Over this long Assention Day weekend, I've got IOzone running some benchmarks. Once those are complete I'll graph and post the data. Below is a sample graph from my first test IOzone run with iozone -Ra -g 64G -n 8B -z -b out.wks.

The main reason behind these benchmarks are not the pretty graphs, which I do love. What we are looking for is a comparison with our existing server infrastructure to ensure the new system will hold up under the load of our on-demand streaming servers.

Once the NAS box is up in the server room, we'll perform additional configuration, tests and tuning. E.g.: Perform these benchmarks over the network from the client machines. Perform them again one the NFS clustering with drbd is setup with a 2nd NAS box. And then we go into production!

There is still much work to be done. Once all the raw data is collected and analyzed, I'll be posting here again for your reading pleasure.

The series:
Building up & testing a new 9TB SATA RAID10 NFSv4 NAS, part I
Building up & testing a new 9TB SATA RAID10 NFSv4 NAS, part II
Building up & testing a new 9TB SATA RAID10 NFSv4 NAS, part III