I''ve heard it said that crossbow performance will decrease when a NIC has run out of ring buffers. Is this true? How would one determine the number of hardware buffers? And are there any general tips that can help us determine what the hardware capabilities are in relationship to large VNIC deployments? In general, how do we appropriately size our use to hardware capability? benr. -- This message posted from opensolaris.org
On 01/09/09 15:39, Ben Rockwood wrote:> I''ve heard it said that crossbow performance will decrease when a NIC > has run out of ring buffers. Is this true? How would one determine the > number of hardware buffers? And are there any general tips that can > help us determine what the hardware capabilities are in relationship > to large VNIC deployments?dladm show-phys -H displays group and ring information for physical devices. # dladm show-phys -H LINK GROUP TYPE NUM-RINGS CLIENTS e1000g2 0 RX 1 -- e1000g0 0 RX 1 e1000g0 e1000g1 0 RX 1 -- e1000g3 0 RX 1 -- nxge0 0 RX 4 vnic12,vnic11 nxge0 1 RX 0 -- nxge1 0 RX 3 vnic22 nxge1 1 RX 1 vnic21 nxge2 0 RX 3 vnic35,vnic34,vnic33,vnic32 nxge2 1 RX 1 vnic31 nxge3 0 RX 4 -- nxge3 1 RX 0 -- # dladm show-vnic LINK OVER SPEED MACADDRESS MACADDRTYPE VID vnic11 nxge0 1000 0:14:4f:da:3e:cc factory, slot 3 0 vnic12 nxge0 1000 0:14:4f:da:3e:cd factory, slot 4 0 vnic21 nxge1 0 0:14:4f:da:3e:d1 factory, slot 1 0 vnic22 nxge1 0 0:14:4f:da:3e:d2 factory, slot 2 0 vnic31 nxge2 0 0:14:4f:da:3e:d8 factory, slot 1 0 vnic32 nxge2 0 0:14:4f:da:3e:d9 factory, slot 2 0 vnic33 nxge2 0 0:14:4f:da:3e:da factory, slot 3 0 vnic34 nxge2 0 0:14:4f:da:3e:db factory, slot 4 0 vnic35 nxge2 0 0:14:4f:da:3e:dc factory, slot 5 0> In general, how do we appropriately size our use to hardware capability? >#dladm create-vnic -H creates vnics with h/w rings. this fails if there are insufficient h/w resources to create the vnic. in the example above, vnic11,12,21,22 were created with the -H flag vnic31-35 were not. -mike
Excellent, I was curious what the -H flag did. Can you make any comments regarding the advantages or disadvantages of using HW vs SW rings? Is there an order-of-magnitude difference? Also, any recommendations on a NIC with a large quantity of rings? I''m supprised that both your and my Intel Pro''s only have 1, whereas your NGE''s have 4. Thanks! benr. -- This message posted from opensolaris.org
Ben Rockwood wrote:> Also, any recommendations on a NIC with a large quantity of rings? I''m supprised that both your and my Intel Pro''s only have 1, whereas your NGE''s have 4.If you''re looking for 1Gb NIC, Intel Zoar(igb) and later NICs have 4(more on latest hardware) hardware rx rings. As to 10GbE, Neterion Xframe II/E has 8 rx rings, and Intel Oplin(ixgbe) has 8 rx rings *exposed*. I guess you were referring to 1Gb version of nxge, which has 4 hardware rings. NGE(nVidia 1GbE) doesn''t expose any hardware rings so far. Thanks, Roamer. -- # telnet (650)-786-6759 (x86759) Connected to Solaris.Sun.COM. login: Lu, Yunsong Last login: January 2, 2007 from beyond.sfbay Yunsong.Lu at Sun.COM v1.04 Since Mon Dec. 22, 2003 [Roamer at Solaris Networking]# cd ..
On 01/09/09 19:27, Yunsong (Roamer) Lu wrote:> Ben Rockwood wrote: > >> Also, any recommendations on a NIC with a large quantity of rings? I''m supprised that both your and my Intel Pro''s only have 1, whereas your NGE''s have 4. >> > If you''re looking for 1Gb NIC, Intel Zoar(igb) and later NICs have > 4(more on latest hardware) hardware rx rings. As to 10GbE, Neterion > Xframe II/E has 8 rx rings, and Intel Oplin(ixgbe) has 8 rx rings *exposed*. > > I guess you were referring to 1Gb version of nxge, which has 4 hardware > rings. NGE(nVidia 1GbE) doesn''t expose any hardware rings so far. >that''s correct... the nxge card on that system has 4x1G interfaces and there are other nxge cards with 1x10G. -mike -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/crossbow-discuss/attachments/20090116/f576e52d/attachment.html>