Hi, I''m trying to perform some (what I consider) basic traffic shaping on our network utilising HTB. I have mostly reused the example on the lartc.org site: tc qdisc add dev eth0 root handle 1: htb default 10 tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit burst 24k tc class add dev eth0 parent 1:1 classid 1:10 htb rate 50mbit ceil 80mbit burst 19k tc class add dev eth0 parent 1:1 classid 1:20 htb rate 1mbit ceil 1mbit tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10 tc qdisc add dev eth0 parent 1:20 handle 20: sfq perturb 10 tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip src X.X.X.X flowid 1:20 So we have a total of 100mbit to be used, the default class 1:10 gets 50mbit and a ceiling of 80mbit whereas my test host X.X.X.X gets only 1mbit in any situation. Some rate limiting is definitely happening, but I am finding the outbound traffic is limited to 2mbit instead of 1mbit. If I change the rate (to say 10mbit) the outbound traffic gets up to again twice the rate (in this case 20mbit). Any thoughts? I have had a look at the tc statistics but it doesn''t appear as I would expect it to. Class 1:10 shows a lot of dropped packets but it is only averaging around 30mbit constantly. On the other hand class 1:20 doesn''t show any dropped packets. Similarly there are no packets marked as overlimit for any class. I occasionally see the tokens for 1:20 go negative... everything is quite strange. Any help would be appreciated. -- Regards, Oliver Hookins
> tc qdisc add dev eth0 root handle 1: htb default 10 > tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit burst 24kDoes the following help? tc qdisc add dev eth0 root handle 1: htb default 1 tc class add dev eth0 parent 1: classid 1:1 htb rate 81mbit burst 24k -- DO4-UANIC
Denis Ovsienko wrote:>>tc qdisc add dev eth0 root handle 1: htb default 10 >>tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit burst 24k > > Does the following help? > tc qdisc add dev eth0 root handle 1: htb default 1 > tc class add dev eth0 parent 1: classid 1:1 htb rate 81mbit burst 24kThat seems a bit backward, limiting the total available bandwidth to enforce child class limits. Or is that how htb works, that the sum of the child classes must add up to no less than the parent class? In any case we have 100mbit available so I don''t want to set it any lower than that, if possible. -- Regards, Oliver Hookins
Oliver Hookins wrote:> Hi, > > I''m trying to perform some (what I consider) basic traffic shaping on > our network utilising HTB. I have mostly reused the example on the > lartc.org site: > > tc qdisc add dev eth0 root handle 1: htb default 10 > tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit burst 24k > tc class add dev eth0 parent 1:1 classid 1:10 htb rate 50mbit ceil > 80mbit burst 19kburst 19k will limit you unless your HZ=1000> tc class add dev eth0 parent 1:1 classid 1:20 htb rate 1mbit ceil > 1mbit > tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10 > tc qdisc add dev eth0 parent 1:20 handle 20: sfq perturb 10 > tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip src > X.X.X.X flowid 1:20 > > So we have a total of 100mbit to be used, the default class 1:10 gets > 50mbit and a ceiling of 80mbit whereas my test host X.X.X.X gets only > 1mbit in any situation. Some rate limiting is definitely happening, but > I am finding the outbound traffic is limited to 2mbit instead of 1mbit. > If I change the rate (to say 10mbit) the outbound traffic gets up to > again twice the rate (in this case 20mbit).Not sure can you show output of tc -s -d class ls dev eth0 while it''s running at 2x speed.> > Any thoughts? I have had a look at the tc statistics but it doesn''t > appear as I would expect it to. Class 1:10 shows a lot of dropped > packets but it is only averaging around 30mbit constantly. On the other > hand class 1:20 doesn''t show any dropped packets. Similarly there are no > packets marked as overlimit for any class.Apart from the 30mbit that would be normal. Andy. I occasionally see the tokens> for 1:20 go negative... everything is quite strange. > > Any help would be appreciated. >
Oliver Hookins wrote:> Denis Ovsienko wrote: > >>> tc qdisc add dev eth0 root handle 1: htb default 10 >>> tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit burst 24k >> >> >> Does the following help? >> tc qdisc add dev eth0 root handle 1: htb default 1 >> tc class add dev eth0 parent 1: classid 1:1 htb rate 81mbit burst 24k > > > That seems a bit backward, limiting the total available bandwidth to > enforce child class limits. Or is that how htb works, that the sum of > the child classes must add up to no less than the parent class? > > In any case we have 100mbit available so I don''t want to set it any > lower than that, if possible. >You haven''t got 100mbit once overheads are accounted for and in this test using 81 instead of 100 won''t affect your bandwidth as all traffic is going to one of the two sub classes. I have IIRC spotted differences (to do with quantum and sharing excess) when parent is > than ceils - it wasn''t the same setup as yours though. Andy.
Andy Furniss wrote:> Oliver Hookins wrote: > >> Hi, >> >> I''m trying to perform some (what I consider) basic traffic shaping on >> our network utilising HTB. I have mostly reused the example on the >> lartc.org site: >> >> tc qdisc add dev eth0 root handle 1: htb default 10 >> tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit burst 24k >> tc class add dev eth0 parent 1:1 classid 1:10 htb rate 50mbit ceil >> 80mbit burst 19k > > > burst 19k will limit you unless your HZ=1000Our HZ is 512.>> tc class add dev eth0 parent 1:1 classid 1:20 htb rate 1mbit ceil >> 1mbit >> tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10 >> tc qdisc add dev eth0 parent 1:20 handle 20: sfq perturb 10 >> tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip src >> X.X.X.X flowid 1:20 >> >> So we have a total of 100mbit to be used, the default class 1:10 gets >> 50mbit and a ceiling of 80mbit whereas my test host X.X.X.X gets only >> 1mbit in any situation. Some rate limiting is definitely happening, >> but I am finding the outbound traffic is limited to 2mbit instead of >> 1mbit. If I change the rate (to say 10mbit) the outbound traffic gets >> up to again twice the rate (in this case 20mbit). > > > Not sure can you show output of tc -s -d class ls dev eth0 while it''s > running at 2x speed.class htb 1:1 root rate 100Mbit ceil 100Mbit burst 24Kb/8 mpu 0b cburst 132644b/8 mpu 0b level 7 Sent 741267447736 bytes 735629628 pkts (dropped 0, overlimits 0) rate 4316188bps 22688pps lended: 489885589 borrowed: 0 giants: 0 tokens: -78 ctokens: 6677 class htb 1:10 parent 1:1 leaf 10: prio 0 quantum 200000 rate 50Mbit ceil 80Mbit burst 19Kb/8 mpu 0b cburst 106440b/8 mpu 0b level 0 Sent 740240463634 bytes 734522966 pkts (dropped 429910165, overlimits 0) rate 4233957bps 22358pps lended: 244988819 borrowed: 489885528 giants: 0 tokens: -293 ctokens: 6636 class htb 1:20 parent 1:1 leaf 20: prio 0 quantum 13107 rate 1Mbit ceil 1Mbit burst 2Kb/8 mpu 0b cburst 2Kb/8 mpu 0b level 0 Sent 755354078 bytes 755048 pkts (dropped 0, overlimits 0) rate 84198bps 321pps backlog 4p lended: 755220 borrowed: 61 giants: 0 tokens: -21517 ctokens: -21517 -- Regards, Oliver Hookins
Oliver Hookins wrote:>> burst 19k will limit you unless your HZ=1000 > > Our HZ is 512. >I don''t know if it makes any difference, but I would have chosen 500 so that it was 2ms. The default now is 250 and with 19k burst that fits the speed you get really well - with 512 it would be around 70meg. I would get rid of the 2 burst settings you use.>> Not sure can you show output of tc -s -d class ls dev eth0 while it''s >> running at 2x speed. > > > class htb 1:1 root rate 100Mbit ceil 100Mbit burst 24Kb/8 mpu 0b cburst > 132644b/8 mpu 0b level 7 > Sent 741267447736 bytes 735629628 pkts (dropped 0, overlimits 0) > rate 4316188bps 22688pps > lended: 489885589 borrowed: 0 giants: 0 > tokens: -78 ctokens: 6677 > > class htb 1:10 parent 1:1 leaf 10: prio 0 quantum 200000 rate 50Mbit > ceil 80Mbit burst 19Kb/8 mpu 0b cburst 106440b/8 mpu 0b level 0 > Sent 740240463634 bytes 734522966 pkts (dropped 429910165, overlimits 0) > rate 4233957bps 22358pps > lended: 244988819 borrowed: 489885528 giants: 0 > tokens: -293 ctokens: 6636 > > class htb 1:20 parent 1:1 leaf 20: prio 0 quantum 13107 rate 1Mbit ceil > 1Mbit burst 2Kb/8 mpu 0b cburst 2Kb/8 mpu 0b level 0 > Sent 755354078 bytes 755048 pkts (dropped 0, overlimits 0) > rate 84198bps 321pps backlog 4p > lended: 755220 borrowed: 61 giants: 0 > tokens: -21517 ctokens: -2151784198 bps - tc means bytes/sec by this, so it''s < 1mbit. Htb uses really long (100 sec) average for its rate so it probably is giving 1mbit. Andy.