Re: Problems migrating from netflow to sFlow: results are low

Christian,

Seems like a lot of variables here... May I suggest another test?

sFlowTrend allows you to switch back and forth instantly between a
minute-by-minute picture of the interface counter trend (derived from
100% accurate hardware counters) and the top-talkers trend (derived
from random packet samples). If all is well, the inbound and
outbound frames/sec numbers should approximately match the inbound
and outbound totals on the "top sources by frames" chart.

This way you can verify the integrity of the sampling source
separately from the other system components (sflowtool, netflow,
database etc.)

For this test, if you only have a few hundred packets per second on
this link then you might try changing the sampling setting to 1-
in-128 or 1-in-32. Packet loss in transit will not be a problem
because sFlowTrend uses the "sample-pool" counter for it's scaling
(so 1% packet loss would just make 1-in-128 equivalent to 1-in-129).

regards,
neil

On Aug 11, 2005, at 1:08 AM, Christian Hammers wrote:

> Hello
>
> Thanks for the quick answers. To Jay: No, we're using only Linux (with
> nprobe) and Cisco to export netflow. The lack of proper netflow export
> was one reason for migrating to sflow :-)
>
> On Wed, Aug 10, 2005 at 10:05:15AM -0700, neil mckee wrote:
>
>> 1. Most Foundry switches only sample inbound traffic on an
>> interface, so if you only enabled sflow on the uplink then you are
>> probably only seeing the traffic in one direction.
>>
> That is not the case with the FastIron Edge series, which I could
> verify.
>
> http://www.foundrynet.com/services/documentation/ecmg/
> Net_Monitoring.html
> * FastIron Edge Switches. FES devices support sFlow packet
> sampling
> for inbound and outbound traffic on sFlow-enabled ports.
> ...
> On these devices, sample data is collected from inbound traffic
> on ports
> enabled for sFlow. Outbound traffic is sampled on the FastIron
> Edge
> Switches only. However, both traffic directions are counted for
> byte and
> packet counter statistics sent to the collector.
>
>
>> 2. sflowtool does not aggregate samples together, even if they are
>> from the same flow. I don't know what program you are using to
>> populate the database, but you might want to check that if two flows
>> with the same key are entered then they end up being added together
>> (rather than the second one replacing the first).
>>
> No, we save everything as-is what sflowtool exports us via netflow.
> The
> final data looks like this (currently very low polling intervals and
> rates):
> +---------------------+-----------------+--------------+---------+
> | time | s | d | dOctets |
> +---------------------+-----------------+--------------+---------+
> | 2005-08-11 00:46:59 | 222.119.133.1XX | 85.197.2.1XX | 202752 |
> | 2005-08-11 00:47:34 | 222.119.133.1XX | 85.197.2.1XX | 67072 |
> | 2005-08-11 00:49:05 | 222.119.133.1XX | 85.197.2.1XX | 744448 |
> | 2005-08-11 00:49:27 | 222.119.133.1XX | 85.197.2.1XX | 744448 |
> | 2005-08-11 00:49:40 | 222.119.133.1XX | 85.197.2.1XX | 744448 |
> | 2005-08-11 00:50:05 | 222.119.133.1XX | 85.197.2.1XX | 744448 |
>
>
>
>> 3. It's helpful to check the frames counter as well as the bytes
>> counter. The frames estimates will converge faster for a given
>> sampling rate, and it is not subject to any differences in the way
>> that bytes are counted. For netflow export, sflowtool uses the ip-
>> len from the ip header to say how many layer3 bytes there were. If
>> your netflow source was reporting the bytes including layer2 headers
>> then there would be a sizable discrepancy. (There are also
>> different ways to interpret udp packets if they are larger than 1518
>> bytes and being fragmented at the IP layer.)
>>
>> You might cross-check using another product (http://www.sflow.org/
>> products/collectors.php). Some of them are free downloads (e.g.
>> sFlowTrend, ntop, pmccact).
>>
> Although our netflow also only exports layer3 and there shouldn't be
> that much large UDP packets, I will try to install another converter.
>
> bye,
>
> -christian-
>
> --
> Christian Hammers WESTEND GmbH | Internet-Business-
> Provider
> Technik CISCO Systems Partner - Authorized
> Reseller
> L|tticher Stra_e 10 Tel
> 0241/701333-11
> ch@westend.com D-52064 Aachen Fax
> 0241/911879
Received on Fri Aug 12 12:50:58 2005