I have a 2Mbit connection, and I would like to distribute that 250k/s
(upstream bandwidth, of course, because I am adding the qdisc on our
internet interface) as follows:
* From our residence students (199.212.55.128/25), min 50k/s,
ceil 200k/s
* From our DMZ network (where our webserver is located,
199.212.53.0/24), min 200k/s, ceil 250k/s
This is the configuration I _think_ should work:
tc qdisc add dev eth8 root handle 1: htb default 11
tc class add dev eth8 parent 1: classid 1:1 htb rate 250kbps
tc class add dev eth8 parent 1:1 classid 1:10 htb rate 200kbps\
ceil 250kbps burst 10k
tc qdisc add dev eth8 parent 1:10 sfq perturb 5
tc filter add dev eth8 protocol ip parent 1: prio 1 u32 match ip\
src 199.212.53.0/24 flowid 1:10
tc class add dev eth8 parent 1:1 classid 1:11 htb rate 50kbps \
ceil 200kbps burst 2k
tc qdisc add dev eth8 parent 1:11 sfq perturb 10
tc filter add dev eth8 protocol ip parent 1: prio 4 u32 match \
ip src 199.212.55.128/25 flowid 1:11
So, our residence students are assigned to classid 1:11. Notice that
1:10, our DMZ, is given prio 1, because I want it to get any extra
bandwidth before our residence. Indeed, residence users are happily
throttled at 200k/s (sustaining it, thanks to the proliferation of p2p
file sharing programs). However, when I hit our webserver (from 3
outside sites with lots of bandwidth), I am only able to transfer at
about 65k/s (peak 85k/s), while the residence students only drop down to
about 160k-180k/s. What I would expect to happen is residence dropping
to 50k/s, and our webserver serving up at 200k/s. (Incidentally I am
monitoring the input/output on the interfaces using iptraf to measure
bandwidth usage.) I am sustaining the transfer for 20-30 seconds before
measuring bandwidth usage.
If I drop the ceil on 1:11 to 150kbps, I am able to get about 100-120k/s
from the web server. And if I again drop the ceil to 100kbps, I get
around 175k/s from the webserver. So clearly my remote sites are able
to pull down the full 200k/s, which is my goal.
So knowing what I want, and my current configuration, can anyone see
what I''m doing wrong?
Thanks,
Jason.
--
Academic Computing Support Specialist
Algoma University College
Sault Ste. Marie, Ontario
705-949-2301 x330 Personal Home Page
http://www.auc.ca http://sault.org
> tc class add dev eth8 parent 1:1 classid 1:10 htb rate 200kbps\ > ceil 250kbps burst 10k > tc filter add dev eth8 protocol ip parent 1: prio 1 u32 match ip\ > src 199.212.53.0/24 flowid 1:10 > > tc class add dev eth8 parent 1:1 classid 1:11 htb rate 50kbps \ > ceil 200kbps burst 2k> So, our residence students are assigned to classid 1:11. Notice that > 1:10, our DMZ, is given prio 1, because I want it to get any extra > bandwidth before our residence. Indeed, residence users are happilyYou did it wrong ! prio 1 attached to filter has to do nothing with prio 1 attached to the class. I have a BIG objectives to the overal filter design in Linux QoS but I can''t do anything with it. Simply add prio 1 to the class definition (last tc line above). And let me know ! ;) devik
Hi Martin. Thanks for your reply. On Sat, 2002-02-02 at 08:14, Martin Devera wrote:> filter design in Linux QoS but I can''t do anything with it. > Simply add prio 1 to the class definition (last tc line above). > And let me know ! ;)In fact, there is no change. Here is my current configuration, with the changes you suggested: tc class add dev eth8 parent 1: classid 1:1 htb rate 250kbps tc class add dev eth8 parent 1:1 classid 1:10 htb rate 200kbps \ ceil 250kbps burst 10k prio 1 tc qdisc add dev eth8 parent 1:10 sfq perturb 5 tc filter add dev eth8 protocol ip parent 1: u32 match ip \ src 199.212.53.0/24 flowid 1:10 tc class add dev eth8 parent 1:1 classid 1:11 htb rate 50kbps \ ceil 200kbps burst 2k prio 4 tc qdisc add dev eth8 parent 1:11 sfq perturb 5 tc filter add dev eth8 protocol ip parent 1: u32 match ip \ src 199.212.55.128/25 flowid 1:11 Monitoring eth8, I see it sending out at 200k/s, all traffic generated from 199.212.55.128/25. This is what I''d expect. When I hit the webserver (on 199.212.53.0/24) hard, eth8 begins to send at 250k/s (it is a 2Mbit uplink), but the portion of that from the webserver is only about 50-90/s, not 200k/s like I''d expect based on id 1:10. Any other thoughts? Regards, Jason. -- Academic Computing Support Specialist Algoma University College Sault Ste. Marie, Ontario 705-949-2301 x330 Personal Home Page http://www.auc.ca http://sault.org
> > Simply add prio 1 to the class definition (last tc line above). > > And let me know ! ;) > > In fact, there is no change. Here is my current configuration, with the > changes you suggested: > > ceil 200kbps burst 2k prio 4default max prio is 3 by the way but it is not problem here ..> Monitoring eth8, I see it sending out at 200k/s, all traffic generated > from 199.212.55.128/25. This is what I''d expect. When I hit the > webserver (on 199.212.53.0/24) hard, eth8 begins to send at 250k/s (it > is a 2Mbit uplink), but the portion of that from the webserver is only > about 50-90/s, not 200k/s like I''d expect based on id 1:10.very interesting ! I''m interested in "tc -s -d class show dev eth8" output in middle of your run. Please take several snapshots, copy into file along with time (I need to know the time difference between them). I''ll look at them and try to deduce the problem. I use configuration like your regulary with no probs so that I''m really interested. devik
On Sun, 2002-02-03 at 15:20, Martin Devera wrote:> very interesting ! I''m interested in "tc -s -d class show dev eth8" output > in middle of your run. Please take several snapshots, copy into file along > with time (I need to know the time difference between them).Okay. Attached is a commented typescript taken. The first invocation of ''tc show'' is before I initiated the remote wgets, the next 2 are during, and the last is after I stopped them. I don''t quite understand all the data, but strictly looking at the rate field, I think it pretty nicely shows what''s happening. (Except on the last reading, which shows a rate of ~40k/s even though the web server''s interface was idle -- I assumed it was just an averaging window issue.)> I''ll look at them and try to deduce the problem. I use configuration like > your regulary with no probs so that I''m really interested.Thanks very much for your help on this, Martin. :) Jason. -- Academic Computing Support Specialist Algoma University College Sault Ste. Marie, Ontario 705-949-2301 x330 Personal Home Page http://www.auc.ca http://sault.org
(tc output below cropped)> class htb 1:11 parent 1:1 leaf 8066: prio 3 rate 400Kbit ceil > 1600Kbit burst 2Kb/8 mpu 0b cburst 3647b/8 mpu 0b quantum 5120 ] > level 0 > rate 192901bps 236pps backlog 36p > > class htb 1:1 root prio 0 rate 2000Kbit ceil 2000Kbit burst 4159b/8 > mpu 0b cburst 4159b/8 mpu 0b quantum 25600 level 3 > rate 191724bps 239pps > > class htb 1:10 parent 1:1 leaf 8065: prio 1 rate 1600Kbit ceil > 2000Kbit burst 10Kb/8 mpu 0b cburst 4159b/8 mpu 0b quantum 20480 > level 0 > rate 1380bps 9ppsNow that I study this a bit more, how can it be that the parent class, 1:1, has a rate that is LESS than the sum of its children''s rates? Either I''ve uncovered a bug, or I''m greatly misunderstanding something. ;) Jason.
> > rate 192901bps 236pps backlog 36p > > rate 191724bps 239pps > > rate 1380bps 9pps > > Now that I study this a bit more, how can it be that the parent class, > 1:1, has a rate that is LESS than the sum of its children''s rates? > Either I''ve uncovered a bug, or I''m greatly misunderstanding something. > ;)The rate discrepancy is ok. Each class is averaged on its own and there is really possible to overshot parent''s rate for a while. It is accounted to the child later. But is seems to me that problem is at your www server. First, the HTB reports no backlog for the www connection - VERY unlikely ! It means that all data has been sent and no new arrived. The same is shown by tokens. They are positive which means that class is not throtling your data. I''m not sure what is going here but absence of backlog is alarming. SFQ can hold up to 128 packets and server which is trying to send more data would create some backlog, wouldn''t it ? devik
On Sun, 2002-02-03 at 17:25, Martin Devera wrote:> But is seems to me that problem is at your www server. First, the > HTB reports no backlog for the www connection - VERY unlikely ! It > means that all data has been sent and no new arrived. > The same is shown by tokens. They are positive which means that > class is not throtling your data.Okay, except if I drop the ceil of 1:11 to 50kbps, I can get about 150-200k/s from the webserver, which tells me that there is no bottleneck on the webserver or on the remote sites pulling the data down. What else could account for the fact that there is no backlog? Jason.
> Okay, except if I drop the ceil of 1:11 to 50kbps, I can get about > 150-200k/s from the webserver, which tells me that there is no > bottleneck on the webserver or on the remote sites pulling the data > down. > > What else could account for the fact that there is no backlog?Only bug. :-| Just to eliminate potential bugs please try to add burst 20k to all your classes (1:1,10,11) and send me statistics again - the one during download is enough. It would be nice to do it like: while true; do tc -s -d class show dev eth8; sleep 1;done > file to get idea what is done during whole time. There is still "brute force" way: run htb with debug 3333333 and do tcpdump of packets. But it would be time consuming way :( By the way, what speed is the machine ? devik
On Mon, 2002-02-04 at 02:01, Martin Devera wrote:> burst 20k to all your classes (1:1,10,11) and send me statistics > again - the one during download is enough. It would be nice to > do it like: > while true; do tc -s -d class show dev eth8; sleep 1;done > fileOkay. I''ve accumulated about 30k worth of data, which I will attach to a separate email just to you (so as not to spam the list).> By the way, what speed is the machine ?It is a Celeron 366, with a load average of about 0.10. Jason.