crossbow discuss - Dec 2007 - Question about interfaces available after crossbow...

Hi,

Talking with some developers who are working on software to
monitor network traffic, they''ve got the 10G problem on their
horizon and trying to work out how to handle it.  There''s a slight
twist on using crossbow, as it is: all of their traffic has the same
IP#/UDP port pairs.  What they''d need to be able to do is perform
classification on data inside the UDP body.  Is that possible with
the current 10G NICs (neptune, etc) and if so, will the interfaces
that crossbow exports (at the programming level) support expressing
that kind of matching?

Thanks,
Darren

Darren,

Not sure if we have any NICs which do classification beyond
layer 4. Although it would be really possible to do this
using our soft classifier.

We need to define some APIs into the pseudo H/W layer of
Crossbow which includes the H/W and S/W implementations. If
the H/W is capable, we use it else we do it in S/W. We will
have some unofficial (not stable) available as part of
Crossbow. Things that we can do as part of a command line
which includes
1) Creating VNICs and flows (including classification)
2) Assigning/modifying attributes like B/W, priority, CPU list to
NICs/VNICs/Flows (any link).
3) Getting some of the stats, real time usage out

We are also looking to expose the VNICs out as interfaces for
MIB so snmp agents can get that info including attributes and
stats etc.

We also need to expose a MAC client API so that clients can
access the MAC layer and register function callbacks and
cookies for packets they want based on classification info
supplied in 1 above.

Anyway, is someone interested in writing up a formal API document
that would be useful. Also exposing the VNICs out via MIB
would a great mini project for someone to volunteer for....

Cheers,
Sunay

Darren Reed wrote:
> Hi,
> 
> Talking with some developers who are working on software to
> monitor network traffic, they''ve got the 10G problem on their
> horizon and trying to work out how to handle it.  There''s a slight
> twist on using crossbow, as it is: all of their traffic has the same
> IP#/UDP port pairs.  What they''d need to be able to do is perform
> classification on data inside the UDP body.  Is that possible with
> the current 10G NICs (neptune, etc) and if so, will the interfaces
> that crossbow exports (at the programming level) support expressing
> that kind of matching?
> 
> Thanks,
> Darren
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss

-- 
Sunay Tripathi
Distinguished Engineer
Solaris Core Operating System
Sun MicroSystems Inc.

Solaris Networking:     http://www.opensolaris.org/os/community/networking
Project Crossbow:       http://www.opensolaris.org/os/project/crossbow

Sunay Tripathi wrote:
> Darren,
>
> Not sure if we have any NICs which do classification beyond
> layer 4. Although it would be really possible to do this
> using our soft classifier.
....
> We need to define some APIs into the pseudo H/W layer of
> Crossbow which includes the H/W and S/W implementations. If
> the H/W is capable, we use it else we do it in S/W. We will
> have some unofficial (not stable) available as part of
> Crossbow.
Only doing classification based on layers 3 and 4 works as long
as what you want to partition is defined by the information you
see in layer 3 & 4.

With various datagram based protocols, there is relevant information
for demuxing the packet flow at the beginning of the data section
inside the packet.

Ideally it would be nice if classification could also work on TCP
data so that you could push a connection to a specific queue
depending on whether or not you see "GET", "POST", etc, at
the start of the HTTP connection.  But maybe that is better left
to those building web servers to direct traffic to the right host
for balancing out load?

Darren

Hi,
i''m considering finding a way integrate discovering, monitoring and
reporting on VNIC and flows from the crossbow project.
My prefered protocols is snmp with MIBs (very used in network world) but
i''m interested as well in using DTrace.
Do you know if MIBs or DTrace scripts exists for crossbow?
Sylvain Quartier
 
 
This message posted from opensolaris.org

Hi Sylvain,

We are almost ready to push out a new set of Crossbow bits on 
OpenSolaris in next 10 days. VNICs should show up as IF in
those bits. You can derive the B/W limit set on the NIC or VNIC
by looking at the link speed in the MIB. Exposing the other
attributes and flows will need a bit more work and we are looking
at it.

BTW, can you describe a little bit more what you are doing and
what other kind of things you need to see. We are also working
on defining our APIs and Dtrace hooks to get you more visibility
into the stack, NICs, VNIC and flows including real time usage,
history etc. So it would be good to work with people like you
to make sure it meets the need.

Cheers,
Sunay

Sylvain Quartier wrote:
> Hi,
> i''m considering finding a way integrate discovering, monitoring
and reporting on VNIC and flows from the crossbow project.
> My prefered protocols is snmp with MIBs (very used in network world) but
i''m interested as well in using DTrace.
> Do you know if MIBs or DTrace scripts exists for crossbow?
> Sylvain Quartier
>  
>  
> This message posted from opensolaris.org
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss

-- 
Sunay Tripathi
Distinguished Engineer
Solaris Core Operating System
Sun MicroSystems Inc.

Solaris Networking:     http://www.opensolaris.org/os/community/networking
Project Crossbow:       http://www.opensolaris.org/os/project/crossbow

On Jan 17, 2008, at 11:29 AM, Sunay Tripathi wrote:
> We are almost ready to push out a new set of Crossbow bits on
> OpenSolaris in next 10 days. VNICs should show up as IF in
> those bits. You can derive the B/W limit set on the NIC or VNIC
> by looking at the link speed in the MIB. Exposing the other
> attributes and flows will need a bit more work and we are looking
> at it.
Note that the VNICs already show-up as regular interfaces in our  
current bits if you have already installed them, with each VNIC having  
its own set of statistics.

Nicolas.

-- 
Nicolas Droux - Solaris Networking - Sun Microsystems, Inc.
droux at sun.com - http://blogs.sun.com/droux

Hi Sunay,

To explain shortly what are doing, technically, at InfoVista:

- We discover a infrastructure environment (networks, systems,
applications) and determine which metrics to collect and the best to
collect them

- We feed our centralized database (marketing call it CMDB, it is based
on oracle OEM) with all these inventory information

- our centralized system launch the data collection and start reports.
It calculate as well advanced KPIs from root metrics

- We create web based reports on the whole infrastructure, with
performance data, trending, prediction ... and real time alerts as well
when we detect bad trending.

Today we support most of network config in the world (this is our
primary market) and we cover many system & apps config as well (we
support all unix, linux & win OS, we collect through SNMP, WMI and
DTrace remotely, we can collect as well from BMC Patrol API and Tivoli
SSM Agent).

 

Concerning metrics in the network environment, the main high level
categories are:

 

The main interest with crossbow are Interface (NIC & VNIC) and all
related to QoS (flows)

Concerning metrics, the very primary requirement are around MIB2
definition:

-    Traffic In/Out (%, bps & Bytes), Error In/Out (%, #). If possible,
discard & broadcast, don''t know if this applicable for crossbow.

-    Concerning flows: very similar to previous, but by application. In
Cisco world, we have as well "queue" info to tell us if the flows is
saturating.

But the challenge is more in the discovery phase. We need some config
information:

-    NIC & VNIC: bandwidth limit, IP / Mac.

-    For VNIC: identify which physical NIC is related to this Virtual
NIC. This is, for example, similar to identify a MPLS interface in a
router, that is logical interface related to a physical interface. In
Cisco world, we identify that using IfIndex of interfaces.

-    For flows: application and limit configured.

-    Also, it will be very good to identify which VNIC is associated to
which Zone. But it is something that InfoVista discovery tool can
discover automatically

 

The best for first step is to support MIB2 that describe all of that and
is standardized in InfoVista.

 

Sylvain Quartier

------------------------------------------------------------------------
--------------------------

Product Management Director, DataCenters & Enterprise Services

www.infovista.com

 

 

-----Original Message-----
From: Sunay Tripathi [mailto:Sunay.Tripathi at Sun.Com] 
Sent: jeudi 17 janvier 2008 19:29
To: Sylvain Quartier
Cc: crossbow-discuss at opensolaris.org
Subject: Re: [crossbow-discuss] Question about interfaces available
after

 

Hi Sylvain,

 

We are almost ready to push out a new set of Crossbow bits on 

OpenSolaris in next 10 days. VNICs should show up as IF in

those bits. You can derive the B/W limit set on the NIC or VNIC

by looking at the link speed in the MIB. Exposing the other

attributes and flows will need a bit more work and we are looking

at it.

 

BTW, can you describe a little bit more what you are doing and

what other kind of things you need to see. We are also working

on defining our APIs and Dtrace hooks to get you more visibility

into the stack, NICs, VNIC and flows including real time usage,

history etc. So it would be good to work with people like you

to make sure it meets the need.

 

Cheers,

Sunay

 

Sylvain Quartier wrote:
> Hi,
> i''m considering finding a way integrate discovering, monitoring
and
reporting on VNIC and flows from the crossbow project.
> My prefered protocols is snmp with MIBs (very used in network world)
but i''m interested as well in using DTrace.
> Do you know if MIBs or DTrace scripts exists for crossbow?
> Sylvain Quartier
>  
>  
> This message posted from opensolaris.org
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss
 

 

-- 

Sunay Tripathi

Distinguished Engineer

Solaris Core Operating System

Sun MicroSystems Inc.

 

Solaris Networking:
http://www.opensolaris.org/os/community/networking

Project Crossbow:       http://www.opensolaris.org/os/project/crossbow

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/crossbow-discuss/attachments/20080118/8d60f695/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 10635 bytes
Desc: image001.png
URL:
<http://mail.opensolaris.org/pipermail/crossbow-discuss/attachments/20080118/8d60f695/attachment.png>