From: Marc Lavine (mlavine@foundrynet.com)
Date: 08/26/02
Hello all,
I've been doing some work on an sFlow collector implementation and
I've noticed some extensibility issues with the existing standard (as
described in RFC 3176). The current standard seems to lack the
ability to gracefully extend the types of data included in sFlow
datagrams without breaking compatibility with existing collector
implementations.
As an example, with the existing datagram design, if a new type of
extended data is defined, introducing it requires changing the
datagram definition. Because the encoding of the extended data types
does not include length information, a collector cannot skip over new
extended data types that it does not recognize. As a result of this,
the datagram version number must be changed when new extended data
types are added in order to avoid having collectors mis-parse
datagrams with the new extended data types. A collector which has not
yet been updated to recognize the new datagram version must discard
the entire datagram because it cannot be sure that it can parse any of
it correctly.
It seems that this could cause considerable hassles for customers when
upgrading agents and collectors as the standard evolves. To address
this problem, it seems to me that the datagram format should be
changed so that extended data types are preceded by a length field.
This would make them sufficiently self-descriptive that a collector
could skip over any ones which are not known to it. There would be
some additional space overhead in doing this, but I think it would be
well worth the improved interoperability over the long term. I think
it would be a good idea to apply this principle of making the data
self-descriptive elsewhere as well. I think that counter samples
should also have a length field added so that new types of counter
samples could be added without breaking compatibility with existing
collectors. (When I say "existing collectors" here, I mean those
collectors which would support this new sFlow version being proposed.)
If this length field is placed at the beginning of the counters_sample
structure, then a corresponding length field can also be added to the
flow_sample structure, which would then provide for the possibility of
adding new sample types in the future while maintaining compatibility
with existing collectors.
Adding length fields to structures as I've described actually provides
for two different types of extensibility. The first type of
extensibility, which I referred to above, involves adding new
structures, identified with new type tags (i.e. adding new branches to
certain unions in the XDR specification of the sFlow datagram format).
The second type of extensibility involves being able to extend the
length of an already-existing structure in a way that need not break
compatibility with collectors which understand an older version of
that structure. As an example, let's say that a new field is
identified that should belong in the extended_gateway structure.
Rather than define a new extended data type just for this new field,
it could simply be added to the end of the existing structure. Since
the structure would include a length field, a collector can process
the portion of the structure that it knows about, and skip over the
rest.
To ensure that changes are only made in a compatible fashion, the
sFlow datagram specification can state that the self-descriptive
structures may be extended in future revisions of the specification by
adding fields to the end of the structures, and that no fields my be
removed or reordered, and that such changes would only be made to
structures which are not included within other structures (eliminating
the generic if_counters structure as a candidate for this type of
extensibility). It can also state that such changes would not require
a change in the type tag associated with the structure, nor in the
sFlow datagram version number.
There is a third type of extensibility that we also have a need for,
and that is the ability to add vendor-specific extensions. While we
certainly think it's good to standardize things that are common across
vendor implementations, different vendors have various kinds of
extensions, which are not standardized, and yet which could be very
useful to include in sFlow samples (as long as someone will build a
collector to utilize them, of course).
To implement this, I propose that a new type tag or two be added to
the list for the extended data types. These type tags would indicate
vendor-specific extensions. The structures for these would include a
length (as proposed above for all extended data type structures), and
would have a vendor-assigned identifier which would indicate which
type of content was present, and then actual content would follow.
For the vendor-assigned identifiers, I propose two different options.
For a vendor to be able to assign their own identifiers, these need to
be constructed based on a globally-unique identifier. The two that
come to mind are an IEEE OUI (Organizationally Unique Identifier),
which appears as the first three bytes of a MAC address, and an SNMP
enterprise OID. Most switch vendors are likely to already have one or
both of these already assigned to them. The sFlow specification could
support one or the other of these types of identifiers, or support
both by defining two extended data types, one for use with each kind
of identifier. As far as possible encodings, an OUI plus a single
additional byte would fit in a single XDR unsigned int, while an SNMP
OID would probably be reasonably compactly encoded as an XDR
variable-length opaque field containing the ASN.1-encoded OID.
Overall, I think these changes will allow for greater interoperability
between different versions of sFlow agents and collectors and should
make sFlow easier for customers to use. I look forward to any
comments on this proposal.
Regards,
Marc Lavine
Foundry Networks, Inc.
This archive was generated by hypermail 2.1.4 : 08/26/02 PDT