«

»

Jan 05

pmacct CentOS 6 with flow-fanout

Sigh, I swear I find all the weird edge cases with every product I touch, hence this blog. The latest being pmacct on CentOS 6 with a flow-fanout

The latest relates to those shiny new ASRs I mentioned earlier and netflow.

So we use Netflow on our border routers to account traffic leaving the network and let people have free traffic within the DC.

We have a few netflow boxes (stats, analytics and accounting) so to spare the load on the routers we use flow-fanout.

In our network ig goes something like this

10.30.30.0/24 on the routers -> 10.30.30.10 Fanout eth1 -> 10.30.31.0/24 eth2 -> Various boxes, one of which is pmacct

Becuase of the way we use pmacct, it needs the SNMP interface index, so we pass the 10.30.30.0/24 address to pmacct

Now this is all well and good on CentOS 5, but were trying to upgrade from that, and when I spun it up on Cent OS 6, i couldn’t get it to work for the life of me. I thought it was ESXi (all our netflow is on one box to save moving data around the network 4 times)

I’ll spare you the long story but its to do with the network stack on CentOS 6, even with the interface in promiscious mode, its rp_filter.

rp filter is responsible for a reverse path check. Because the flow-fanout box rewrite part of the packet header, it looks like a spoofed packet and although you can see the traffic on the interface, it doesn’t make it to pmacct.

The fix is to put it into a nice mode with:

echo 2 >  /proc/sys/net/ipv4/conf/all/rp_filter

And viola! Netflow data, sweet sweet netflow data