Re: Host sFlow

A new draft of the sFlow Host Structures document has been posted:
http://www.sflow.org/sflow_host-draft3.txt

The open source Host sFlow agent fully implements this specification on Windows, Linux, Xen and XenServer platforms and has been deployed on a number of different data center networks:
http://host-sflow.sourceforge.net/
The Host sFlow agent is available in source code form and can be easily ported to additional platforms.

The sflowtool command line utility now decodes host sflow structures and can be used as a starting point for incorporating these additional structures in sFlow analysis software. The source code for sflowtool is available at:
http://www.inmon.com/technology/sflowTools.php

The project has reached a point where it makes sense to finalize the sFlow Host Structures specification. Please review the specification and send any comments to the list. The goal is to publish a final draft by the end of July.

The sFlow Host Structures specification creates a framework for application level performance monitoring. Once the specification has been finalized future work to define data structures for reporting on the performance of popular networked services should be relatively straightforward.

So far, we have seen interest in instrumenting the following protocols:
HTTP
NFS
Hadoop
Memcached
SIP

If anyone has a particular interest in one of these applications, or would like to suggest other applications, please start a thread to discuss the counters and transaction attributes that should be exported by an sFlow agent monitoring the service.

Finally, the Host sFlow agent software makes use of DNS-SD to configure sFlow monitoring:
http://blog.sflow.com/2010/06/dns-sd.html

DNS-SD works well for configuring large numbers of sFlow agents throughout the network. Standardizing the set of attributes associated with sFlow configuration in DNS-SD would create a useful alternative to CLI and SNMP based configuration of sFlow that would allow an entire data center of switches, routers and servers to be configured using a single, simple mechanism.

On May 19, 2010, at 5:16 PM, Peter Phaal wrote:

> The Host sFlow agent now supports Windows performance monitoring (in addition to Linux):
> http://host-sflow.sourceforge.net/
>
> Metrics (based on libvirt) have been defined for reporting virtual machine performance. A new draft of the Host sFlow specification contains the new structures:
> http://www.sflow.org/sflow_host-draft2.txt
>
> The Host sFlow agent can export the virtual machine metrics on Xen/XenServer platforms. Installing the recently released Open vSwitch 1.0.0 virtual switch along with Host sFlow on XenServer gives complete visibility into physical and virtual network and system performance and makes a good platform to try out the technology.
>
> For directions on configuring sFlow on the Open vSwitch see:
> http://blog.sflow.com/2010/05/configuring-open-vswitch.html
>
> On Apr 3, 2010, at 12:10 PM, Peter Phaal wrote:
>
>> Current trends toward virtualization and convergence tightly link networking and system performance and blur the line between the network and the servers. For example, virtualization places switching and routing functions on servers. Monitoring the network in this environment requires that the servers are also monitored. Similarly anyone concerned with application and server performance must now also be concerned about network performance since application features (such as virtual machine migration) can significantly affect and be affected by network performance.
>>
>> Currently, performance monitoring of servers and applications is highly fragmented. Each server vendor, operating system vendor and application developer creates agents and software for performance monitoring, none of which interoperate. Monitoring performance of a server might require the same (or similar but incompatible metrics) be monitored more than once since each management application requires its own agent. In addition, there may be other agents monitoring different hardware and software elements within the server.
>>
>> The development of performance and application monitoring in sFlow requires progress on two parallel tracks:
>> 1. specifying the additional structures to be used when exporting performance metrics in sFlow
>> 2. implementing sFlow agents that export the metrics in order to allow application developers and system administrators to deploy agents and gain experience.
>>
>> The draft Host sFlow specification describes some basic sFlow structures for monitoring host performance and lays a foundation for future work to extend sFlow to monitor virtual machines and application services (e.g. Xen, Hyper-V, VMware, HTTP, NFS, memcached, Hadoop etc):
>> http://www.sflow.org/sflow_host-draft1.txt
>>
>> The Host sFlow sourceforge project is building an open source Host sFlow agent that can easily be ported to a wide variety of operating systems:
>> http://host-sflow.sourceforge.net/
>>
>> Initial prototypes of the Host sFlow agent have been running on over 1000 Windows and Linux servers.
>>
>> For server and application management, the combination of sFlow in the top of rack switch and Host sFlow is particularly compelling. Servers maintain many useful performance counters, but don't typically have hardware support for packet sampling so have a very limited view their network I/O. The top of rack switch has hardware packet sampling support and sees all the traffic in and out of the servers. Host sFlow exports the counters that are currently stranded on the server, allowing a performance management system to put together the network and system performance for each server by combining data from the switch and the servers.
Received on Thu Jul 15 12:03:09 2010