Over the summer we at xtendx moved to a new data center at Green.ch, and I setup a new two-node 9 TB DRBD + ext4 + NFS4 NAS. I've also setup a Cacti monitoring system to graph key metrics and alert us to problems. One metric that the Cacti community has not gotten around to addressing is the relatively new and shiny NFSv4 clients and servers.

To address that omission, I started on my first Cacti graph template project. What the Cacti community does have is an NFS v3 client graphing template, so I used this as a base. Here is my preliminary result:

The data aquisition is performed on the target machine with a simple Bash (/usr/local/bin/cacti-nfs4.sh) script that is executed by snmpd. The script:

NFS=/proc/net/rpc/nfs
proc="read write commit open open_confirm open_named_att_dir \
  open_downgrade close set_attr fsinfo renew set_clientid confirm lock \
  lock_test unlock access get_attr loopup lookup_root remove rename \
  link symlink create pathconf statfs readlink readdir server_pas \
  delegreturn getacl setacl"

i=4;

for a in $proc; do
#       echo -n "$a.value "
        grep proc4 $NFS \
                | cut -f $i -d ' ' \
                | awk '{print $1}'
        i=$(expr $i + 1)
done

To configure snmpd to call the script and report the result, I've added the below to /etc/snmp/snmpd.conf and then restarted the snmpd daemon. (Now is probably a good time to mention that this is all on Debian Lenny, so if you are on a different platform YMMV.)

extend .1.3.6.1.4.1.2021.66 nfs_client /bin/sh /usr/local/bin/cacti-nfs4.sh

Once this is done the configuration can be confirmed with snmpwalk, like this below. The output not only should be there, but also match the output of cacti-nfs4.sh

ballito:/home/stu# !snmpwalk
snmpwalk -v2c -c public  ballito.be .1.3.6.1.4.1.2021.66.4
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.1 = STRING: "1209434"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.2 = STRING: "46246622"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.3 = STRING: "46218746"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.4 = STRING: "52265481"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.5 = STRING: "69"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.6 = STRING: "0"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.7 = STRING: "2"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.8 = STRING: "52197430"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.9 = STRING: "46346973"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.10 = STRING: "2"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.11 = STRING: "0"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.12 = STRING: "1"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.13 = STRING: "1"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.14 = STRING: "0"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.15 = STRING: "0"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.16 = STRING: "0"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.17 = STRING: "3260450"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.18 = STRING: "42080094"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.19 = STRING: "47829475"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.20 = STRING: "1"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.21 = STRING: "2357"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.22 = STRING: "89"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.23 = STRING: "0"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.24 = STRING: "0"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.25 = STRING: "161"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.26 = STRING: "1"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.27 = STRING: "27924"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.28 = STRING: "0"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.29 = STRING: "46133"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.30 = STRING: "3"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.31 = STRING: "140977"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.32 = STRING: "0"
UCD-SNMP-MIB::ucdavis.66.4.1.2.10.110.102.115.95.99.108.105.101.110.116.33 = STRING: "0"
ballito:/home/stu# 

With the data being properly collected and reported by SNMP, it's now time to configure Cacti to record and graph it all. nfsstat (which is at the core of the Bash script) reports 35 different metrics for NFS v4. For my tastes, that is way too many metrics for a single graph. I've defined an abridged Cacti graph template with 14 values for now, and intended to squeeze that down to 10 in the upcoming weeks. That said, the template has all but one data source template included. In the end, I think having two graphs may make sense: An everything graph, and a "values of interest" graph.

Here is the graph template itself: cacti_graph_template_ucdnet_-_nfs4_client.xml

Please feel free to let me know if you have any issues with it, have made improvements worth posting, or found it useful and want to buy me a beer!

Add a comment

Posted by Stu Thompson on 16 October 2010 10:10:00 CEST #