Re: Clarifying frame_length in sFlow packet headers

From: Marc Lavine (mlavine@foundrynet.com)
Date: 06/26/03

  • Next message: Peter Phaal: "Encoding unknown addresses in sFlow datagrams"

    Hi Peter,

    Thanks for posting this proposal (in response to some issues I'd raised in a
    private email). See comments below...

    ----- Original Message -----
    From: "Peter Phaal" <peter.phaal@inmon.com>
    To: <sflow@sflow.org>
    Sent: Wednesday, June 25, 2003 10:53 AM
    Subject: [sFlow] Clarifying frame_length in sFlow packet headers

    > There is some potential ambiguity in the current definition of
    frame_length
    > in the sampled_header structure:
    >
    > /* Raw Packet Header */
    > /* opaque = flow_data; enterprise = 0; format = 1 */
    >
    > struct sampled_header {
    > header_protocol protocol; /* Format of sampled header */
    > unsigned int frame_length; /* Original length of packet before
    > sampling.
    > Note: For a layer 2 header_protocol,
    > length is total number of
    octets
    > of data received on the
    network
    > (excluding framing bits but
    > including FCS octets) */
    > opaque header<>; /* Header bytes */
    > }
    >
    > According to this definition, the length is defined for layer 2 protocols
    as
    > the total octets on the wire. This definition fully accounts for the
    > bandwidth due to layer 2 traffic. There are a few situations where there
    can
    > be problems with this definition:
    >
    > 1. In a situation where layer 2 information (such as VLAN tags or FCS
    > octets) is stripped by hardware, an sFlow agent either has to reconstruct
    > the stripped layers and include them in the header<>, or adjust the
    > frame_length to account for the stripped bytes. If no corrective action is
    > taken then an sFlow decoder may overestimate the bytes attributed to
    higher
    > layer protocols.

    So my understanding of the main issue here, is that an sFlow decoder, having
    decoded a sampled packet header up to some "layer 3" header, may want to
    determine how many bytes of payload data should be attributed to those layer
    3 and above protocol(s), and due to an unknown number of bytes having been
    stripped from the original packet, the decoder can't determine where the
    layer 2 payload data ends (which in turn, may indicate where the layer 3
    payload data ends).

    > 2. Trailing information, such as the layer 2 FCS octets can cause
    problems.
    > For example, an encapsulated frame may or may not have the FCS bytes on
    the
    > wire. Since a packet decoder would assume that the frame included a FCS,
    it
    > would incorrectly subtract 4 bytes from payload, underestimating the layer
    3
    > packet size. The agent could add 4 bytes to the packet length. However,
    this
    > would cause the link utilization to be miscalculated.
    >
    > 3. There may be proprietary or ambiguous lower layer encapsulations
    > associated with tunnels that must be stripped if the header<> is to be
    fully
    > decodeable. Stripping these headers causes the same problems as 1.

    Yes, this is a good point.

    > The addition of a new "stripped" counter would seem to resolve these
    > problems. It allows the frame_length to continue to be defined as the
    octets
    > on the wire, but provides information on any leading and trailing
    > information that were stripped before the header<> was extracted. The
    > proposed change is as follows:
    >
    > /* Raw Packet Header */
    > /* opaque = flow_data; enterprise = 0; format = 1 */
    >
    > struct sampled_header {
    > header_protocol protocol; /* Format of sampled header */
    > unsigned int frame_length; /* Original length of packet before
    > sampling.
    > Note: For a layer 2 header_protocol,
    > length is total number of
    octets
    > of data received on the
    network
    > (excluding framing bits but
    > including FCS octets) */

    One might also note that in some cases (e.g. due to hardware limitations
    with different encapsulations), it might not be possible to report a
    completely accurate frame length, but the agent should try to come as close
    as possible.

    > unsigned int stripped; /* The number of octets removed from
    the
    > packet before extracting the
    header<>.
    > Trailing protocol information, such
    as
    > FCS information, must always be
    > stripped.
    > e.g. a non-encapsulated 802.3 packet
    > would
    > always have stripped=4

    The definition of the "stripped" field has some ambiguities. The notion of
    "trailing protocol information" should probably be clarified a bit. Perhaps
    calling it "trailing encapsulation data" would help slightly? As for the
    requirement that it be stripped, one could try to clarify this by stating
    that trailing encapsulation data corresponding to any leading encapsulations
    that were stripped must also be stripped (to keep things symmetrical), and
    trailing encapsulation data for the outermost protocol layer included in the
    sampled header must be stripped.

    Also, a non-encapsulated 802.3 packet would always have stripped>=4 (rather
    than =4), since VLAN tag information might have been stripped off in
    addition to the FCS.

    > Any leading encapsulations that are
    > ambiguous, or not one of the
    standard
    > header_protocol values must be
    > stripped.

    Perhaps it would be better to say "outer encapsulations" rather than
    "leading encapsulations" since an encapsulation might have both leading and
    trailing portions?

    > opaque header<>; /* Header bytes */
    > }
    >
    > Any comments?

    The stripped field seems like a reasonable solution to the issue of
    accurately counting octets on the wire, while still being able to determine
    the payload length for higher-layer protocols. I have some issues related
    to the stripping of VLAN tags, however, but I'll send out a separate message
    on that topic later.

    Regards,
    Marc



    This archive was generated by hypermail 2.1.4 : 06/26/03 PDT