Initial DRBD Sync – 9TB in 4.5 days
After much blood, sweat and tears getting xtendx up and running in our new data center with Green.ch this summer, I finally had time to complete a major “high availability” aspect of our new file server platform. Our basic architecture is fairly simple: a couple of application servers backed by the network attached storage.

- May: Build NAS 0 (thanks to my wife, Robyn, too!)
- Early June: Install NAS 0 into data center, copy over production data
- Mid-June: Put NAS 0 into production as a simple ext4 + NSFv4 file server
- July: Build NAS 1 (Again, a thanks to my wife)
- July + August: Install NAS 1 into data center, configure as a HA NAS: drbd + ext4 + NSFv4.
- September: Copy data to NAS 1, take NAS 1 into production
- Late-September: configure NAS 0 as secondary node in a HA NAS: drbd + ext4 + NSFv4.
- Early-October: Initialize DRBD synchronization.
Kicking off the block-level disk synchronization was a big deal. These servers are a running production system and it is paramount that existing service delivery was not impacted. At first I had left the DRBD sync rate unaltered, which I believe effectively means ‘fast as you can’. This quickly resulted in poor read times for production applications, so I kicked it down dramatically while the sync was in progress with the drbdsetup
command:
sudo drbdsetup /dev/drbd1 syncer -r 24M
After fooling around with various rates, I settled on 24M. It’s much lower than the system could theoretically synchronize at, but that is not the goal. This production value is also now configured in /etc/drbd.d/global_common.conf
:
common {
#snip
syncer {
# rate after al-extents use-rle cpu-mask verify-alg csums-alg
rate 24M;
}
}
Interestingly, the synchronization rate does not match disk IO as measured by Cacti via SNMP; in fact it is roughly half. (I have no idea why.) As you can see below, it took a solid 4.5 days for the synchronization of a single 9TB device to complete:


A bonus to this work on a production system is that the DRBD sync reserved as a real-world production load test. I now know with a respectable degree of certainty what our NAS is capable of, and when it is approaching stressed.