Broadcom Corp. Bhushan Kanekar http://www.broadcom.com/ Broadcom Corp. Nolan Leake Cumulus Networks Inc. Peter Phaal InMon Corp. December 2015 sFlow Broadcom Peak Buffer Utilization Structures Copyright Notice Copyright (C) Broadcom Corp. (2015). All Rights Reserved. Abstract This memo describes sFlow version 5 structures for exporting Broadcom buffer utilization statistics from switch ASICs. Table of Contents 1. Overview ...................................................... 1 2. Discussion .................................................... 2 3. sFlow Datagram Extensions ..................................... 2 4. References .................................................... 3 5. Author's Addresses ............................................ 3 1. Overview This document describes additional structures that allow an sFlow agent on a Broadcom ASIC based switch to export peak buffer utilization statistics gathered using the Buffer Statistics Tracking (BST) feature [1]. sFlow version 5 is an extensible protocol that allows the addition of new data structures without impacting existing collectors. This document does not change the sFlow version 5 protocol [2], it simply defines additional, optional, data structures that an sFlow agent can use to report on buffer allocation. FINAL Broadcom Corp. [Page 1] FINAL Broadcom Peak Buffer Utilization Structures December 2015 2. Discussion The Memory Management Unit (MMU) is on-chip logic that manages how the on-chip packet buffer is organized. Buffer Statistics Tracking (BST) is a feature that enables tracking the usage of these buffers. It includes snapshot views of peak utilization of the on-chip buffer memory across queues, ports, priority group, service pools and the entire chip. The MMU has on-chip memory dedicated to packet buffers. This memory is organized as N byte cells. The Network Operating System (NOS) programmatically allocates the buffers into the following: * For Ingress: Service pool, Per-port Service pool, Priority Group, Port. * For Egress: Service pool, Port Service pool, and Queues. There are separate queues and accounting for unicast and multicast traffic. Traffic based on CoS/DSCP is allocated to a particular queue. Some buffer allocations are guaranteed and some use shared pools and headroom with an allocation scheme. The sFlow export of BST statistics reports peak utilization of the buffers in use on a per- queue basis. These counters are intended for: * Trend analysis * Microburst detection * Network planning 3. sFlow Datagram Extensions Peak buffer utilizations are calculated based on BST buffers counts, e.g.: ingress_uc_pc = 100 * (ingress_uc_buffers_used / ingress_uc_buffer_allocated) The values are encoded as a percentage[2]: /* Percentage expressed in hundredths of a percent (e.g., 100 = 1%). If a percentage value is unknown then use the value -1. */ typedef int percentage; The following structures are used to report peak buffer usage: /* Device level buffer utilization */ /* buffers_used metrics represent peak since last export */ /* opaque = counter_data; enterprise = 4413; format = 1 */ FINAL Broadcom Corp. [Page 2] FINAL Broadcom Peak Buffer Utilization Structures December 2015 struct bst_device_buffers { percentage uc_pc; /* unicast buffers percentage utilization */ percentage mc_pc; /* multicast buffers percentage utilization */ } /* Port level buffer utilization */ /* buffers_used metrics represent peak buffers used since last export */ /* opaque = counter_data; enterprise = 4413; format = 2 */ struct bst_port_buffers { percentage ingress_uc_pc; /* ingress unicast buffers utilization */ percentage ingress_mc_pc; /* ingress multicast buffers utilization */ percentage egress_uc_pc; /* egress unicast buffers utilization */ percentage egress_mc_pc; /* egress multicast buffers utilization */ percentage egress_queue_uc_pc<8>; /* per egress queue unicast buffers utilization */ percentage egress_queue_mc_pc<8>; /* per egress queue multicast buffers utilization*/ } /* Selected egress queue */ /* opaque = flow_data; enterprise = 4413; format = 1 */ struct extended_bst_egress_queue { unsigned int queue; /* eqress queue number selected for sampled packet */ } 4. References [1] "Broadcom Broadview", https://www.broadcom.com/products/ethernet- communication-and-switching/switching/broadview [2] Phaal, P. and Lavine, M., "sFlow Version 5", http://www.sflow.org/sflow_version_5.txt, July 2006 5. Author's Address Bhushan Kanekar Broadcom Corp. 3151 Zanker Road San Jose, CA 95134 Nolan Leake Cumulus Networks Inc. 185 E. Dana Street Mountain View, CA 94041 Peter Phaal InMon Corp. 1 Sansome Street, 35th Floor San Francisco, CA 94104 FINAL Broadcom Corp. [Page 3]