We run a Hosting farm behind a bridge/iptables firewall setup running Gentoo with kernel 2.4.20-gentoo-r6, connected to a dual 15Mbps international internet pipe / , as this: Net Pipe --------- eth1 Bridge/Firewall eth0 -------- Internal Hosting Network lately we have been looking at htb to somehow control excessive usage from the users behind, but in our implementation there seems to be an error or something wrong on the setup, this is the test script i''m using, i know it''s very rough but i think it should do the work. tc qdisc del dev eth1 root tc qdisc add dev eth1 root handle 1: htb default 10 tc class add dev eth1 parent 1: classid 1:1 htb rate 98Mbit ceil 98Mbit tc class add dev eth1 parent 1:1 classid 1:10 htb rate 90Mbit ceil 90Mbit tc class add dev eth1 parent 1:1 classid 1:11 htb rate 2Mbit ceil 2Mbit tc class add dev eth1 parent 1:1 classid 1:12 htb rate 4Mbit ceil 4Mbit tc filter add dev eth1 protocol ip parent 1:0 prio 1 u32 match ip src $server_ip flowid 1:11 I intend to limit $server_ip to 2Mbit max traffic ow, the problem is after i run the script htb seems to ignore the limit and traffic for the client stays in over 3mbit. but after a while of running with the htb active the server owner complains that the loading times of pages hosted on the server skyrocket and that ssh access becomes sluggish. Normally that server has about 4/5 Mbit/s of outgoing traffic measured by the iptables/mrtg script, doing a: #tc -s -d class show dev eth1 shows: class htb 1:11 parent 1:1 prio 0 quantum 26214 rate 2Mbit ceil 2Mbit burst 2621b/8 mpu 0b cburst 2621b/8 mpu 0b level 0 Sent 23592359 bytes 26524 pkts (dropped 1579, overlimits 0) rate 315631bps 352pps backlog 96p lended: 26428 borrowed: 0 giants: 0 tokens: -3 ctokens: -3 class htb 1:1 root rate 98Mbit ceil 98Mbit burst 64212b/8 mpu 0b cburst 64212b/8 mpu 0b level 7 Sent 66766024 bytes 97843 pkts (dropped 0, overlimits 0) rate 889284bps 1291pps lended: 0 borrowed: 0 giants: 0 tokens: 1 ctokens: 1 class htb 1:10 parent 1:1 prio 0 quantum 200000 rate 90Mbit ceil 90Mbit burst 58970b/8 mpu 0b cburst 58970b/8 mpu 0b level 0 Sent 43271713 bytes 71415 pkts (dropped 0, overlimits 0) rate 573411bps 938pps lended: 71415 borrowed: 0 giants: 0 tokens: 1 ctokens: 1 class htb 1:12 parent 1:1 prio 0 quantum 52428 rate 4Mbit ceil 4Mbit burst 2620b/8 mpu 0b cburst 2620b/8 mpu 0b level 0 Sent 0 bytes 0 pkts (dropped 0, overlimits 0) lended: 0 borrowed: 0 giants: 0 tokens: 1 ctokens: 1 Showing trafic in excess of 3.5 Mbit/s and coinciding with a mrtg graphic. From my limited experience i would say that somehow my mrtg is measuring traffic well before it passes thru htb (which seems imposible from what i''ve read). i take the measurement on the iptables FORWARD chain: iptables -N $server_ip-in iptables -N $server_ip-out iptables -A $server_ip-in -j RETURN iptables -A $server_ip-out -j RETURN iptables -A FORWARD -s $server_ip -j $server_ip-out iptables -A FORWARD -d $server_ip -j $server_ip-in and to make the actual measurement: iptables -nvxL $server_ip-in iptables -nvxL $server_ip-out Resuming, how can i effectively test if and how well htb it''s doing the job ? Help will be appreciated. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
It seems you have hit timer innacuracy issues: http://www.docum.org/stef.coene/qos/faq/cache/40.html Rubens On Tue, 27 Jan 2004, Gerardo Arceri wrote:> We run a Hosting farm behind a bridge/iptables firewall setup running > Gentoo with kernel 2.4.20-gentoo-r6, connected to a dual 15Mbps > international internet pipe / , as this: > > Net Pipe --------- eth1 Bridge/Firewall eth0 -------- Internal Hosting > Network > > lately we have been looking at htb to somehow control excessive usage from > the users behind, but in our implementation there seems to be an error or > something wrong on the setup, > this is the test script i''m using, i know it''s very rough but i think it > should do the work. > > tc qdisc del dev eth1 root > tc qdisc add dev eth1 root handle 1: htb default 10 > tc class add dev eth1 parent 1: classid 1:1 htb rate 98Mbit ceil 98Mbit > tc class add dev eth1 parent 1:1 classid 1:10 htb rate 90Mbit ceil 90Mbit > tc class add dev eth1 parent 1:1 classid 1:11 htb rate 2Mbit ceil 2Mbit > tc class add dev eth1 parent 1:1 classid 1:12 htb rate 4Mbit ceil 4Mbit > tc filter add dev eth1 protocol ip parent 1:0 prio 1 u32 match ip src > $server_ip flowid 1:11 > > I intend to limit $server_ip to 2Mbit max traffic ow, the problem is after > i run the script htb seems to ignore the limit and traffic for the client > stays in over 3mbit. > but after a while of running with the htb active the server owner > complains that the loading times of pages hosted on the server skyrocket > and that ssh access becomes sluggish. > Normally that server has about 4/5 Mbit/s of outgoing traffic measured by > the iptables/mrtg script, doing a: > #tc -s -d class show dev eth1 > shows: > > class htb 1:11 parent 1:1 prio 0 quantum 26214 rate 2Mbit ceil 2Mbit burst > 2621b/8 mpu 0b cburst 2621b/8 mpu 0b level 0 > Sent 23592359 bytes 26524 pkts (dropped 1579, overlimits 0) > rate 315631bps 352pps backlog 96p > lended: 26428 borrowed: 0 giants: 0 > tokens: -3 ctokens: -3 > > class htb 1:1 root rate 98Mbit ceil 98Mbit burst 64212b/8 mpu 0b cburst > 64212b/8 mpu 0b level 7 > Sent 66766024 bytes 97843 pkts (dropped 0, overlimits 0) > rate 889284bps 1291pps > lended: 0 borrowed: 0 giants: 0 > tokens: 1 ctokens: 1 > > class htb 1:10 parent 1:1 prio 0 quantum 200000 rate 90Mbit ceil 90Mbit > burst 58970b/8 mpu 0b cburst 58970b/8 mpu 0b level 0 > Sent 43271713 bytes 71415 pkts (dropped 0, overlimits 0) > rate 573411bps 938pps > lended: 71415 borrowed: 0 giants: 0 > tokens: 1 ctokens: 1 > > class htb 1:12 parent 1:1 prio 0 quantum 52428 rate 4Mbit ceil 4Mbit burst > 2620b/8 mpu 0b cburst 2620b/8 mpu 0b level 0 > Sent 0 bytes 0 pkts (dropped 0, overlimits 0) > lended: 0 borrowed: 0 giants: 0 > tokens: 1 ctokens: 1 > > Showing trafic in excess of 3.5 Mbit/s and coinciding with a mrtg graphic. > > From my limited experience i would say that somehow my mrtg is measuring > traffic well before it passes thru htb (which seems imposible from what > i''ve read). i take the measurement on the > iptables FORWARD chain: > > iptables -N $server_ip-in > iptables -N $server_ip-out > iptables -A $server_ip-in -j RETURN > iptables -A $server_ip-out -j RETURN > iptables -A FORWARD -s $server_ip -j $server_ip-out > iptables -A FORWARD -d $server_ip -j $server_ip-in > > and to make the actual measurement: > iptables -nvxL $server_ip-in > iptables -nvxL $server_ip-out > > Resuming, how can i effectively test if and how well htb it''s doing the > job ? > > > Help will be appreciated. > > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ >_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
You were right thanks for the tip, i suspected there was some clock issue involved, now it works perfectly, 2500Kbit limit is not passed by a single byte... superb, thanks again.! On Wed, 28 Jan 2004 00:20:13 -0200 (BRST), <rubens@etica.net> wrote:> > It seems you have hit timer innacuracy issues: > http://www.docum.org/stef.coene/qos/faq/cache/40.html > > Rubens > > > On Tue, 27 Jan 2004, Gerardo Arceri wrote: > >> We run a Hosting farm behind a bridge/iptables firewall setup running >> Gentoo with kernel 2.4.20-gentoo-r6, connected to a dual 15Mbps >> international internet pipe / , as this: >> >> Net Pipe --------- eth1 Bridge/Firewall eth0 -------- Internal Hosting >> Network >> >> lately we have been looking at htb to somehow control excessive usage >> from >> the users behind, but in our implementation there seems to be an error >> or >> something wrong on the setup, >> this is the test script i''m using, i know it''s very rough but i think it >> should do the work.Using M2, Opera''s revolutionary e-mail client: http://www.opera.com/m2/ _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
> > It seems you have hit timer innacuracy issues: > http://www.docum.org/stef.coene/qos/faq/cache/40.html >Recompiled the kernel with the PSCHED_CPU modification and now tc -s -d class show dev eth1 shows that the server is capped exactly at 312Kpbs (2500Kbit). BUT... read below quote..>> From my limited experience i would say that somehow my mrtg is >> measuring >> traffic well before it passes thru htb (which seems imposible from what >> i''ve read). i take the measurement on the >> iptables FORWARD chain: >> >> iptables -N $server_ip-in >> iptables -N $server_ip-out >> iptables -A $server_ip-in -j RETURN >> iptables -A $server_ip-out -j RETURN >> iptables -A FORWARD -s $server_ip -j $server_ip-out >> iptables -A FORWARD -d $server_ip -j $server_ip-in >> >> and to make the actual measurement: >> iptables -nvxL $server_ip-in >> iptables -nvxL $server_ip-out >>That mrtg measurement still shows 412 Kbps, how could it be. doesn''t seem plausible that iptables reads the packets before the packet scheduler, since it works at a lower level closer to actual hardware. What''s the explanation for this ? Thanks in advance.! Using M2, Opera''s revolutionary e-mail client: http://www.opera.com/m2/ _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
> It seems you have hit timer innacuracy issues: > http://www.docum.org/stef.coene/qos/faq/cache/40.htmlWell, I''ve tried this on a vanilla 2.4.24 kernel but was not able to load sched_htb anymore. The system was a P4 1700MHz - wich should support it. I''m also experiencing HTB overlimiting as I describe here at the list a while ago. Regards, Mike. -- GMX ProMail (250 MB Mailbox, 50 FreeSMS, Virenschutz, 2,99 EUR/Monat...) jetzt 3 Monate GRATIS + 3x DER SPIEGEL +++ http://www.gmx.net/derspiegel +++ _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/