From: Marc Lavine (mlavine@foundrynet.com)
Date: 10/01/02
Peter,
This looks fine. I agree that there's some extra overhead for collectors to
track multiple sequence number streams, but hopefully, this won't be
burdensome. Regarding the issue of a distributed agent implementation
causing an increased number of sFlow datagrams to be sent, that would likely
happen only when data rates are relatively low, such that non-full sFlow
datagrams would be sent as a result of a timer tick. In the scenario that
I'm thinking of, I believe things could be optimized to mostly avoid that,
however, doing so might result in some samples being delivered out of order.
Do you think that collectors should be prepared to deal with such cases by
doing some buffering and reordering of out-of-order samples, or do you feel
that's too much of a burden?
Marc
----- Original Message -----
From: Peter Phaal <peter_phaal@inmon.com>
To: 'Marc Lavine' <mlavine@foundrynet.com>; <sflow@sflow.org>
Sent: Wednesday, September 25, 2002 11:58 AM
Subject: RE: [sFlow] Distributed agent implementation
> Marc,
>
> I can see there may be switch architectures in which it is not feasible to
> bring all the samples together into a single sFlow datagram stream. In
this
> case the sub-agent-id field makes sense.
>
> The inclusion of the agent IP address in the sFlow datagram makes sending
> from sub-agents particularly easy. Different sub-agents can send datagrams
> using different IP addresses and still maintain their association with the
> single "master" agent_address for the whole device.
>
> However, I do think that if at all possible it is worth bringing samples
> together into a single stream. This improves scalability of the whole
sFlow
> system by reducing the total number of sFlow datagrams and reducing the
> number of sFlow stream that a collector needs to track.
>
> I've added the change with suggested descriptive language in:
> http://www.sflow.org/drafts/draft6/SFLOW-DATAGRAM.txt
>
> The section relating to sub-agents is as follows:
>
> /* Header information for sFlow version 5 datagrams
>
> The sub-agent field is used when an sFlow agent is implemented on a
> distributed architecture and where it is impractical to bring the
> samples to a single point for transmission.
>
> However, it is strongly recommended that wherever possible the
sub-agent
> mechanism not be used. If multiple processors are available within a
> device
> the various tasks associated with creating flow and counter samples can
> be
> distributed among the processors. However, the agent should be
> architected
> so that all the samples are marshalled into a single datagram stream.
The
> final marshalling task involved very little processing, but has
important
> benefits in making the overall sFlow system scalable. By reducing the
> number of UDP packets and packet streams, the protocol overheads
> associated
> with sFlow are significantly reduced at the receiver.
>
> Each sFlowDataSource must be associated with only one sub-agent. The
> association between sFlowDataSource and sub-agent must remain
> constant for the entire duration of an sFlow session. */
>
> struct sample_datagram_v5 {
> address agent_address /* IP address of sampling agent,
> sFlowAgentAddress. */
> unsigned int sub_agent_id; /* Used to distinguishing between
datagram
> streams from separate agent sub
> entities
> within an device. */
> unsigned int sequence_number; /* Incremented with each sample datagram
> generated by a sub-agent within an
> agent. */
> unsigned int uptime; /* Current time (in milliseconds since
> device
> last booted). Should be set as close
to
> datagram transmission time as
possible.
> Note: While a sub-agents should try
and
> track the global sysUptime
value
> a receiver of sFlow packets
must
> not assume that values are
> synchronised between
sub-agents.
> */
> sample_record samples<>; /* An array of sample records */
> }
>
> Thoughts?
>
> Peter
>
> > Some network switches have multiple CPUs which may be capable of
> > independently performing sFlow sampling for different blocks
> > of traffic
> > moving through the switch. It seems desirable to allow for
> > the possibility
> > of having multiple CPUs within an agent operate independently
> > in performing
> > the sampling, but the current sFlow datagram design has a barrier to
> > accomplishing that, namely the datagram sequence number:
> >
> > struct sample_datagram_v5 {
> > address agent_address /* IP address of sampling agent,
> > sFlowAgentAddress. */
> > unsigned int sequence_number; /* Incremented with each
> > sample datagram
> > generated */
> >
> > Having a single sequence number "stream" for a given agent mandates
> > centralized sequence number allocation, which can interfere
> > with having
> > independent CPUs perform sampling. To address this, I
> > suggest adding an
> > additional field to provide an opaque "sub-agent" id (for
> > lack of a better
> > term), and have the sequence numbers be interpreted only in
> > relation to that
> > id. This would provide for multiple sequence number streams
> > as needed.
> > Thus one might have something like this:
> >
> > struct sample_datagram_v5 {
> > address agent_address /* IP address of sampling agent,
> > sFlowAgentAddress. */
> > unsigned int sub_agent_id; /* Opaque sub-agent id. Used for
> > distinguishing between sequence
> > numbers streams from separate
> > sampling entities within
> > an agent. */
> > unsigned int sequence_number; /* Incremented with each
> > sample datagram
> > generated by the
> > specified sub-agent */
>
>
This archive was generated by hypermail 2.1.4 : 10/01/02 PDT