Since rebuilding my server, I have been fighting a traffic shaping problem. The symptoms are: a) HTB classes consistently exceeded their specified ceilings. b) HTB classes reported a large number of ''giants'' in the output of "shorewall show tc". Example (I acually saw this class reach the maximum uplink capacity at one point): class htb 1:130 parent 1:1 leaf 130: prio 3 quantum 1500 rate 76000bit ceil 230000bit burst 1537b/8 mpu 0b overhead 0b cburst 1614b/8 mpu 0b overhead 0b level 0 Sent 559018700 bytes 75324 pkt (dropped 0, overlimits 0 requeues 0) rate 269288bit 3pps backlog 0b 0p requeues 0 lended: 53963 borrowed: 21361 giants: 90174 tokens: -26688 ctokens: -14783 I finally traced it to our old friend "checksum offloading". With the Xen 3 package included with OpenSuSE 10.2, it is no longer necessary to run ethtool to get communications to work. *But*, if you don''t run "ethtool -K eth0 tx off" in your domUs, traffic shaping in Dom0 *doesn''t work*. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
On Mittwoch 11 April 2007, Tom Eastep wrote:> Since rebuilding my server, I have been fighting a traffic shaping problem. > The symptoms are: > > a) HTB classes consistently exceeded their specified ceilings. > b) HTB classes reported a large number of ''giants'' in the output of > "shorewall show tc". > > Example (I acually saw this class reach the maximum uplink capacity at one > point): > > class htb 1:130 parent 1:1 leaf 130: prio 3 quantum 1500 rate 76000bit ceil > 230000bit burst 1537b/8 mpu 0b overhead 0b cburst 1614b/8 mpu 0b overhead > 0b level 0 > Sent 559018700 bytes 75324 pkt (dropped 0, overlimits 0 requeues 0) > rate 269288bit 3pps backlog 0b 0p requeues 0 > lended: 53963 borrowed: 21361 giants: 90174 > tokens: -26688 ctokens: -14783 > > I finally traced it to our old friend "checksum offloading". With the Xen 3 > package included with OpenSuSE 10.2, it is no longer necessary to run > ethtool to get communications to work. *But*, if you don''t run "ethtool -K > eth0 tx off" in your domUs, traffic shaping in Dom0 *doesn''t work*.Not only traffic shaping, also connection tracking can fail. (Tom, you remember my FTP issue?) Also a *LOT* of TCP retransmission happens because of wrong checksums. So better switch the checksum offloading off, regardless of traffic shaping. Just my 2 cents, Alex ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Alexander Wilms wrote:> > Not only traffic shaping, also connection tracking can fail. (Tom, you > remember my FTP issue?) Also a *LOT* of TCP retransmission happens because of > wrong checksums. So better switch the checksum offloading off, regardless of > traffic shaping. > > Just my 2 cents,Thanks, Alex. On a related note, SuSE users can simplify the configuring of checksum offloading. In the domU''s eth0 config file (found in /etc/sysconfig/network/) you can simply add the following: ETHTOOL_OPTIONS=''-K eth0 tx off'' -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV