Recently I found BGP Expert site, which is a nice Internet routing reference. Once a month the site gathers a list of all the allocated IP addresses around the world from the respective authorities and dumps them into a database. The database is queryable via the Addresses Per Country page.

...the number of IP addresses used in every country of the world. The numbers are generated from information published by the Regional Internet Registries (AfriNIC, APNIC, ARIN, LACNIC and RIPE NCC) published on their FTP servers.

Using this query page, I saved the results from 1990, 2000 and 2009 to a text file, imported into MS Excel, imported again into Google Docs, and finally added a Google Docs Map widget. Here are the results:

(Note: The colors are relative to each individual map.)

China is on the rise!

Add a comment

Posted by Stu Thompson on 29 August 2010 11:39:00 CEST #

Here is the quick-n-dirty on how to configure HP ProCurve series switch ports for dynamic LACP. My direct experience so far is with the HP ProCurve 2510G-24 and the HP ProCurve 2910al-24, but it should translate to similar switches. From other examples I've seen around the Intertubes, the differences you may encounter will be with the port names.

In the below example I am configuring a 4-port LACP trunk. That said, this exact configuration would work for two 2-port trunks.

Here we go: First, log into your switch. Then follow these series of commands:

 HP2910al> 
 HP2910al> config
 HP2910al(config)# int 1-4 disable
 HP2910al(config)# int 1-4 lacp passive
 HP2910al(config)# int 1-4 enable
 HP2910al(config)# 

Next, check the status of the ports. If you have no cables connected then it should look like this.

 HP2910al(config)# show lacp

                           LACP

   PORT   LACP      TRUNK     PORT      LACP      LACP
   NUMB   ENABLED   GROUP     STATUS    PARTNER   STATUS
   ----   -------   -------   -------   -------   -------
   1      Passive   1         Down      No        Success
   2      Passive   2         Down      No        Success
   3      Passive   3         Down      No        Success
   4      Passive   4         Down      No        Success

 HP2910al(config)# 

If you do have cables connected, and the attached network node is properly configured, then this is what you should see after executing the show lacp command.:

 HP2910al(config)# show lacp

                           LACP

   PORT   LACP      TRUNK     PORT      LACP      LACP
   NUMB   ENABLED   GROUP     STATUS    PARTNER   STATUS
   ----   -------   -------   -------   -------   -------
   1      Passive   Dyn1      Up        Yes       Success
   2      Passive   Dyn1      Up        Yes       Success
   3      Passive   Dyn1      Up        Yes       Success
   4      Passive   Dyn1      Up        Yes       Success

 HP2910al(config)# 

Dynamic LACP configuration can be done only in the ProCurve command line interface. Static LACP trunks can be configured at the CLI or the ProCurve's menu system. For an example of this read How to configure Ethernet Bonding with LACP on Debian 5 Linux.

Responses[2]

Posted by Stu Thompson on 25 August 2010 11:09:00 CEST #

Multiple Tomcat instances on Debian Linux? Yup, it can be done with a little bit of configuration magic. The below example assumes that you have multiple IPs already configured on your server and want to run each Tomcat instance on a single unique IP over port 80.

• Don't install a Tomcat package via apt-get or whatever. Besides the packages having a long list of problems with anything but the most trivial setup, it does not help you get multiple instances up and running. Instead download the tar files and unpack into instance specific directories. I personally run each instance with its own user in said user's home directory. E.g.:

  /home/tomcat0
  /home/tomcat0/apache-tomcat-7.0.0
  /home/tomcat1
  /home/tomcat1/apache-tomcat-7.0.0
  /home/tomcat2
  /home/tomcat2/apache-tomcat-7.0.0
  
  

• In conf/server.xml, configure the Server's port attribute to an instance specific value. I use a simple equation: the last for bytes of the IP address plue 8000. E.g.: 10.20.30.40 would use port 8040. The port is used by the bin/shutdown.sh script to signal Tomcat to shutdown. It only listens to the localhost and not the specific IP configured elsewhere.

      <Server port="8058" shutdown="SHUTDOWN"/>

• Also in conf/server.xml is the Connector, which will need configuring. In this example we have the default HTTP Connector. You'll want a unique IP for each instance, which is configured with the address atribute. And if doing port redirection then set the proxyPort attribute too. (See bottom of post for more on this.)

      <Connector port="8080" 
                 proxyPort="80" 
                 address="10.0.0.1"
                 protocol="HTTP/1.1"
                 redirectPort="8443"
               />

I've not bothered with setting up instance specific init.d scripts. For better and worse, I run my instances with manual start up and shutdown. Why? We have 'hot stand-by' servers and prefer our fail-over process to be manually instigated. If anyone one there writes some instance specific scripts I would be glad to post them here.

Regardless, the above configuration will get you up and running with multiple Tomcats on a single box!

(If you are reading this, then you may very well be interested in Port redirection for multiple Tomcat instances on linux 2.6 with iptables too.)

Add a comment

Posted by Stu Thompson on 12 August 2010 22:29:00 CEST #

"The" wife and I went on holiday to Hvar, Croatia earlier this month. We stayed at a quant little flat in town halfway up the karst hillside with an amazing view. So, having brought my camera, tripod and Arduino-based intervalometer, I setup my kit and made a few short clips.

This is my best results yet! That said, I did learn (and relearn) a few lessons for next time:

• Must put the camera into full manual mode. Any automatic settings (speed, aperture, etc.) result in flickering in the video. Semi-automatic modes are not good enough.
• Manually focus once and leave it. The auto focus takes a variable amount of time (leading to inconsistant intervals) and different focal lengths from frame to frame.
• The tripod, a cheap aluminum model, with vertical extension is too wobbly. The movement shows up when zoomed in with the 135mm lens. Keep the vertical extension collapsed.
• Stop using the 'small & coarse' JPEG image quality setting. The compression leaves unsightly artifacts in large, low contrast areas of the photos, like sky & cloud backgrounds.

One thing that kind of bugs me about the Canon is the camera's system of storing photos in sub folders with only 100 stuff each. This is probably because the image file name number maxes out at 9999, another irritating problem because movies can easily be the composite of 10,000+ frames. To resolve both of these issues, I've written a little Bash script that copies the stuff into a single target folder and renames them starting from 00001 (a requirement for FFmpeg).

#!/bin/bash
set -x
id=0
for dir in raw/*CANON
do
    echo "Processing directory $dir"
    for file in $dir/*
    do
        echo "Processing file $file"
        id=$(($id+1))
        name="$(printf "%04d" $id)" 
        
        cp -v $file "orig/$name.jpg"
    done
done

After some processing (resizing, namely) I ran the below FFmpeg command used to compose the movie from the stuff. It uses h264, my favorite video codec, and decent quality settings.

manoa:orig stu$ /opt/local/bin/ffmpeg -f image2 -i %05d.jpg -vcodec libx264 -f mp4
   -flags +loop+mv4 -cmp 256 -partitions +parti4x4+parti8x8+partp4x4+partp8x8+partb8x8
   -me_method hex -subq 7 -trellis 1 -refs 5 -bf 0 -flags2 +mixed_refs -coder 0 -me_range 16 
   -g 75 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -qmin 10 -qmax 51 -qdiff 4  -r 25 -an 
   -b 1280kb -y hvar.avi
FFmpeg version UNKNOWN, Copyright (c) 2000-2009 Fabrice Bellard, et al.
  configuration: --enable-pthreads --enable-libx264 --enable-libmp3lame --enable-libfaad
    --enable-libfaac --disable-ffplay --disable-ffserver --disable-static --enable-shared 
   --enable-nonfree --enable-gpl --enable-postproc --enable-hwaccel=mpeg2_vaapi 
   --enable-hwaccel=mpeg4_vaapi --enable-hwaccel=h263_vaapi --enable-hwaccel=wmv3_vaapi 
   --prefix=/opt/local --extra-cflags=-I/opt/local/include 
   --extra-ldflags=-L/opt/local/lib
  libavutil     50. 3. 0 / 50. 3. 0
  libavcodec    52.35. 0 / 52.35. 0
  libavformat   52.38. 0 / 52.38. 0
  libavdevice   52. 2. 0 / 52. 2. 0
  libswscale     0. 7. 1 /  0. 7. 1
  libpostproc   51. 2. 0 / 51. 2. 0
  built on Dec 15 2009 17:01:07, gcc: 4.0.1 (Apple Inc. build 5493)
Input #0, image2, from '%05d.jpg':
  Duration: 00:07:05.96, start: 0.000000, bitrate: N/A
    Stream #0.0: Video: mjpeg, yuvj420p, 1280x720 [PAR 72:72 DAR 16:9], 25 tbr, 25 tbn, 25 tbc
[libx264 @ 0x1028600]using SAR=1/1
[libx264 @ 0x1028600]using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.1 Cache64
[libx264 @ 0x1028600]profile Baseline, level 3.1
Output #0, mp4, to 'hvar.avi':
    Stream #0.0: Video: libx264, yuv420p, 1280x720 [PAR 1:1 DAR 16:9], q=10-51, 1280 kb/s, 25 tbn, 25 tbc
Stream mapping:
  Stream #0.0 -> #0.0
Press [q] to stop encoding
frame=10649 fps=  5 q=16938494.0 Lsize=   73318kB time=425.96 bitrate=1410.0kbits/s    
video:73233kB audio:0kB global headers:0kB muxing overhead 0.115394%
[libx264 @ 0x1028600]frame I:162   Avg QP:28.94  size: 82562
[libx264 @ 0x1028600]frame P:10487 Avg QP:31.99  size:  5875
[libx264 @ 0x1028600]mb I  I16..4: 31.0%  0.0% 69.0%
[libx264 @ 0x1028600]mb P  I16..4:  2.4%  0.0%  0.7%  P16..4: 32.4%  5.3%  2.2%  0.0%  0.0%    skip:56.7%
[libx264 @ 0x1028600]final ratefactor: 29.67
[libx264 @ 0x1028600]coded y,uvDC,uvAC intra:36.9% 27.2% 12.7% inter:6.6% 2.1% 0.1%
[libx264 @ 0x1028600]ref P L0  48.7% 18.1% 16.8%  7.9%  8.4%
[libx264 @ 0x1028600]kb/s:1408.4
manoa:orig stu$ 

Once the video was uploaded to youtube.com, I added the sound with YouTube's AudioSwap functionality. That feature is very convenient for adding audio to these movies. It sorts the available audio clips into a logical set of categories, has a search function and a "find sound tracks about the same length as my video" option.

Responses[1]

Posted by Stu Thompson on 11 August 2010 20:50:00 CEST #

Over the past two months my company (xtendx AG) moved our servers to a new data center at green.ch. One of the primary motivations for this move was to gain access to a multiple gigabit per second Internet link. Each of our production streaming servers have either 2 or 4 channels ethernet bonding configuration with LACP. Once they were configured I set out to test their capacity and validate the entire design and configuration.

To that end, I installed iperf onto each of our servers. One box was configured as a server. Two others were configured as clients. Because of the method LACP uses to split up the traffic, it is impossible or merely very difficult to setup a server-to-server that uses more than 1Gbps. Generally, LACP accomplish multiple link speeds by splitting the individual client-server traffic over the two links on a client-server address pair basis.

Our setup here is fairly simple: three servers, each with either a 2-channel or 4-channel LACP ethernet bonding setup connected to a lone HP ProCurve 2510G-24 switch. The switch was manually configured to place the ports into a dynamic LACP bond. The bonds themselves are configured with the kernel module parameters mode=4 miimon=100 max_bonds=4 xmit_hash_policy=1

Running iperf in server mode. Note that iperf uses port 5001 by default, so adjust your firewalling solution if necessary.

iperf -s -i 2

Running iperf in client mode. This was done on two physically separate machines.

cat /dev/zero |  iperf -c svr.example.com -t 2400 -i 2

Yup, looks good. There was the possibility that both clients would have come in on the same link. This is possible because the decision about which channel to use is based upon a the source and destination addresses. It is also by design--don't fret! Simply using a different server for one of the clients would resolve the issues.

Responses[4]

Posted by Stu Thompson on 09 August 2010 14:31:00 CEST #

Over the past month I've been converting an existing pair of NFS/ext4 file servers (live & hot-spare with regular rsync-based synchronization) to a two-node Class C DRBD high availability cluster. It's been a learning experience, to say the least. And while the DRBD documentation is excellent, there were some concepts and tasks that did not immediately sink in. The below notes are for my future reference, and anyone who is also new to DRBD.

• Making a DRBD resource mount on boot is not trivial. i've left my fstab with noauto in the configuration. Why? The thought of a node rebooting, becoming stale, and then automatically setting itself to the primary scares me!
  
  
• A DRBD resource that is a Class C secondary is not mountable. you'll get errors like:

  root@umhlanga:/home/stu# mount /mnt/data
  mount: block device /dev/drbd1 is write-protected, mounting read-only
  mount: Wrong medium type

• If you want a DRBD to come up from a cold boot to be primary, then setting up a heartbeat is a requirement. But you don't have to do this for a basic, first step HA cluster.
• Add the startup timeouts options to the configuration file. if your kit is at a data center, without it you may end up making a site visit because of a hung boot. it'll sit there until done. My /etc/drbd.conf snippet:

  common {
    	protocol C;
  	startup {
  		# wfc-timeout degr-wfc-timeout outdated-wfc-timeout wait-after-sb;
  		wfc-timeout 600;
  		degr-wfc-timeout 600;
  		outdated-wfc-timeout 600;
  	}
  }

• A fundamental task to perform is querying the state of the drbd resource(s). There are two ways to do this. Example:

  stu@umhlanga:~$ cat /proc/drbd 
  version: 8.3.7 (api:88/proto:86-91)
  GIT-hash: ... build by root@umhlanga, 2010-07-28 11:28:28
   1: cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown C r----
      ns:0 nr:0 dw:0 dr:0 al:0 bm:6 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:9765272500
  stu@umhlanga:~$ 
  stu@umhlanga:~$ sudo drbd-overview
  [sudo] password for stu: 
    1:r0  WFConnection Primary/Unknown UpToDate/DUnknown C r---- /mnt/data ext4 9.0T 1.5T 7.5T 17% 
  stu@umhlanga:~$ 
  
  

  This output contains lots of important information. The details of each method and the output can be found on DRBD's documentation page 'Checking DRBD Status'

• The high level steps required to bring up a DRBD resource:
  1. kernel starts
  2. network stats
  3. DRBD starts
  4. bring up DRBD resource
  5. make primary (if it is the primary)
  6. mount resource
  And presto! The DRBD resource is ready for use.
  
  
• After having created a resource, and rebooting, you get this message with drbdadm up resource-name, don't follow the instructions in the error message.

  drbdadm up r0 
  1: Failure: (124) Device is attached to a disk (use detach first)
  Command 'drbdsetup 1 disk /dev/sdb /dev/sdb internal --set-defaults 
      --create-device' terminated with exit code 10
  

• If you, like me, are exporting the mounted file system with NFS then be careful with the auto-mount configuration in /ets/fstab.

Add a comment

Posted by Stu Thompson on 08 August 2010 12:57:00 CEST #

In mid-June I installed Debian Lenny onto an old HP DL380 of ours. Since then there has been six of these Generic CACHE Level-2 Generic Error messages.

MCE 0
HARDWARE ERROR. This is *NOT* a software problem!
Please contact your hardware vendor
CPU 6 BANK 3 TSC 1e600ed894af4
ADDR 1719edeb0 
MCG status:
MCi status:
Error enabled
MCi_ADDR register valid
MCA: Generic CACHE Level-2 Generic Error
STATUS 942000420001010a MCGSTATUS 0

Shortly after installing Debian, I also dropped in a second quad core CPU and doubled the RAM to 8GB. Coincidence? Probably not. Then again, I don't have ready access to the old RHEL4 system logs. (They are with our old hosting vendor.)

What exactly does this mean? Is it a recoverable condition? Is it a warning sign of future problems to come?

Spooky.

Add a comment

Posted by Stu Thompson on 05 August 2010 14:17:00 CEST #