Dear all... If we are using TBF, we can reduce "buffer" and "limit" to minimize latency. What should we do in SFQ? I tried to use SFQ, but the latency goes up to 10.000 to 20.000 milisecond in case of overloaded. Can someone help me? I''m using cbq.init, and here is my configuration: Parent class: DEVICE=eth0,10Mbit,1Mbit RATE=64Kbit WEIGHT=6.4Kbit PRIO=5 LEAF=tbf BUFFER=3000 LIMIT=3000 BOUNDED=yes ISOLATED=no In child class: DEVICE=eth0,10Mbit,1Mbit RATE=16Kbit WEIGHT=1.6Kbit PRIO=5 PARENT=0500 LEAF=sfq BOUNDED=no ISOLATED=no PERTURB=5 QUANTUM=3000 RULE=10.0.0.201 Regards, Adi Nugroho
On Tue, Dec 18, 2001 at 03:07:37PM +0800, Adi Nugroho wrote:> Dear all... > > If we are using TBF, we can reduce "buffer" and "limit" to minimize latency. > What should we do in SFQ?You can''t really. SFQ has a fixed queue length of 128.> I tried to use SFQ, but the latency goes up to 10.000 to 20.000 milisecond > in case of overloaded.SFQ is no substitute for prioritization, but this is ridiculous. Can you show the output of cbq.init? You can do this by looking into the script and turning the logfile on. I think cbq.init is misconfiguring things. Regards, bert -- http://www.PowerDNS.com Versatile DNS Software & Services http://www.tk the dot in .tk Netherlabs BV / Rent-a-Nerd.nl - Nerd Available - Linux Advanced Routing & Traffic Control: http://ds9a.nl/lartc
> > I tried to use SFQ, but the latency goes up to 10.000 to 20.000 > > milisecond in case of overloaded. > > SFQ is no substitute for prioritization, but this is ridiculous. Can you > show the output of cbq.init? You can do this by looking into the script and > turning the logfile on. > > I think cbq.init is misconfiguring things.When I started with TC, I also tried the cbq.init script. It''s very limited in what you can do. Once you know TC, you will realize that cbq.init is too limited and that''s doing some things wrong because the situation is much more complicated. An example. You can select tbf as a leaf qfdisc for a cbq class. But the qdisc is allready shaping, so attaching a tbf qdisc is not needed. It can disturb the setup and can result in some strange behaviour. Stef -- stef.coene@docum.org More QOS info : http://docum.org/ Title : "Using Linux as bandwidth manager"
On Tuesday 18 December 2001 16:03, bert hubert wrote:> > I think cbq.init is misconfiguring things. >Dear Sir, As your suggestion, I have made all rule by hand. Bellow are my script, one for cbq (tbf+sfq), and one for htb (+sfq) But the latency was still so high (sometimes more than 20.000 ms if overloaded) From statistic, the overlimit packet is high, but there is no dropped packet Thank you for your help. Regards, Adi Nugroho ================Script for cbq (tbf+sfq) ================dev=eth1 DEVBWDT=10Mbit DEVWGHT=1Mbit tc qdisc del dev eth1 root tc qdisc add dev $dev root handle 1:0 cbq\ bandwidth $DEVBWDT avpkt 1000 cell 8 tc class add dev $dev parent 1:0 classid 1:1 cbq \ bandwidth $DEVBWDT rate $DEVBWDT weight $DEVWGHT \ prio 8 allot 1514 cell 8 maxburst 20 avpkt 1000 #parent class tc class add dev eth1 parent 1:1 classid 1:64 cbq \ bandwidth 10Mbit rate 64Kbit weight 6.4Kbit prio 8\ allot 1514 cell 8 maxburst 20 avpkt 1000 bounded tc qdisc add dev eth1 parent 1:64 tbf \ rate 64Kbit buffer 3000 limit 3000 #Child class #1 tc class add dev eth1 parent 1:64 classid 1:164 cbq \ bandwidth 10Mbit rate 16Kbit weight 1.6Kbit prio 5 \ allot 1514 cell 8 maxburst 20 avpkt 1000 tc qdisc add dev eth1 parent 1:164 sfq \ perturb 5 quantum 4500 tc filter add dev eth1 parent 1:0 protocol ip \ prio 100 u32 match ip dst 10.0.0.222 flowid 1:164 #Child class #2 tc class add dev eth1 parent 1:64 classid 1:264 cbq \ bandwidth 10Mbit rate 16Kbit weight 1.6Kbit prio 5 \ allot 1514 cell 8 maxburst 20 avpkt 1000 tc qdisc add dev eth1 parent 1:264 sfq \ perturb 5 quantum 4500 tc filter add dev eth1 parent 1:0 protocol ip \ prio 100 u32 match ip dst 10.0.0.73 flowid 1:264 #Child class #3 tc class add dev eth1 parent 1:64 classid 1:364 cbq \ bandwidth 10Mbit rate 16Kbit weight 1.6Kbit prio 5 \ allot 1514 cell 8 maxburst 20 avpkt 1000 tc qdisc add dev eth1 parent 1:364 sfq \ perturb 5 quantum 4500 tc filter add dev eth1 parent 1:0 protocol ip \ prio 100 u32 match ip dst 10.0.0.30 flowid 1:364 ======================Script for htb (+sfq) ======================tc qdisc add dev eth1 root handle 1: htb tc class add dev eth1 parent 1: classid 1:1 htb rate 64Kbit burst 15k tc class add dev eth1 parent 1:1 classid 1:10 htb rate 16Kbit ceil 64Kbit burst 15k tc class add dev eth1 parent 1:1 classid 1:20 htb rate 16Kbit ceil 64Kbit burst 15k tc class add dev eth1 parent 1:1 classid 1:30 htb rate 16Kbit ceil 64Kbit burst 15k tc qdisc add dev eth1 parent 1:10 handle 10: sfq perturb 10 tc qdisc add dev eth1 parent 1:20 handle 20: sfq perturb 10 tc qdisc add dev eth1 parent 1:30 handle 30: sfq perturb 10 u32="tc filter add dev eth1 protocol ip parent 1:0 prio 1 u32" $u32 match ip dst 10.0.0.222 flowid 1:10 $u32 match ip dst 10.0.0.73 flowid 1:20 $u32 match ip dst 10.0.0.30 flowid 1:30
On Tue, Dec 18, 2001 at 06:27:51PM +0800, Adi Nugroho wrote:> On Tuesday 18 December 2001 16:03, bert hubert wrote: > > > > I think cbq.init is misconfiguring things. > > > > Dear Sir, > > As your suggestion, I have made all rule by hand. > Bellow are my script, one for cbq (tbf+sfq), and one for htb (+sfq) > > But the latency was still so high (sometimes more than 20.000 ms if > overloaded)Lower maxburst for cbq and burst for htb.> tc class add dev eth1 parent 1:1 classid 1:10 htb rate 16Kbit ceil 64Kbit > burst 15k15k is ~8 seconds! Regards, bert -- http://www.PowerDNS.com Versatile DNS Software & Services http://www.tk the dot in .tk Netherlabs BV / Rent-a-Nerd.nl - Nerd Available - Linux Advanced Routing & Traffic Control: http://ds9a.nl/lartc
Abbreviations:
IFI internal interface
IPI IP address of internal interface
NMI netmask for the internal interface
IFE1, IFE2 external interfaces
IPE1, IPE2 external IP address
NWE1, NWE2 external network address
NME1, NME2 mask for the external network (number of bits, like in /24)
BRD1, BRD2 broadcast address for external network
GWE1, GWE2 gateway for external interface
I am tyring to setup a gw with a cable modem and a dialup connection.
Following the nano.txt, i cant figure out how to define ppp0
ppp0 Link encap:Point-to-Point Protocol
inet addr:166.50.32.206 P-t-P:166.50.32.1 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:4 errors:2 dropped:0 overruns:0 frame:0
TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
collisions:0
RX bytes:40 (40.0 b) TX bytes:92 (92.0 b)
since there is no broadcast or network address that i can tell.
what do i put for these variables..
thx
I am not tyring to route outbound traffice through ppp0 just recieve traffic on that interface. while still being connected to my cable provider.. and yes i have a dynamic ip on ppp0 and a static for my cable.. i think its duable..
Destination Gateway Genmask Flags Metric Ref Use Iface
166.50.32.1 0.0.0.0 255.255.255.255 UH 0 0 0 ppp0
176.30.3.192 0.0.0.0 255.255.255.192 U 0 0 0 eth2
10.2.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 176.30.3.193 0.0.0.0 UG 0 0 0 eth2
eth2 Link encap:Ethernet HWaddr
inet addr:176.30.3.38 Bcast:176.30.3.255 Mask:255.255.255.192
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:98373 errors:0 dropped:0 overruns:0 frame:0
TX packets:116675 errors:0 dropped:0 overruns:0 carrier:0
collisions:1015
RX bytes:76086367 (72.5 Mb) TX bytes:97127829 (92.6 Mb)
ppp0 Link encap:Point-to-Point Protocol
inet addr:166.50.33.32 P-t-P:166.50.32.1 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:4 errors:1 dropped:0 overruns:0 frame:0
TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
collisions:0
RX bytes:40 (40.0 b) TX bytes:61 (61.0 b)
the gw with the lowest metric determines which connection will work..
i cannot add in both gw and make both connections work at the same time..
that is my problem..
On Tue, 18 Dec 2001, Christoph Simon wrote:
> On Tue, 18 Dec 2001 14:00:06 -0500 (EST)
> mok2 <mok2@armageddonnet.com> wrote:
>
> >
> > I am not tyring to route outbound traffice through ppp0 just recieve
> > traffic on that interface.
> > while still being connected to my cable provider..
> >
> > and yes i have a dynamic ip on ppp0 and a static for my cable..
>
> In that case, I''m not sure to understand your problem. You are not
> going to have any multipath route; it a standard setup.
>
> --
> Christoph Simon
> ciccio@kiosknet.com.br
> ---
> ^X^C
> q
> quit
> :q
> ^C
> end
> x
> exit
> ZZ
> ^D
> ?
> help
> .
>
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/
>
On Tue, 18 Dec 2001 13:15:45 -0500 (EST) mok2 <mok2@armageddonnet.com> wrote:> Abbreviations: > IFI internal interface > IPI IP address of internal interface > NMI netmask for the internal interface > IFE1, IFE2 external interfaces > IPE1, IPE2 external IP address > NWE1, NWE2 external network address > NME1, NME2 mask for the external network (number of bits, like in /24) > BRD1, BRD2 broadcast address for external network > GWE1, GWE2 gateway for external interface > > I am tyring to setup a gw with a cable modem and a dialup connection. > Following the nano.txt, i cant figure out how to define ppp0 > > ppp0 Link encap:Point-to-Point Protocol > inet addr:166.50.32.206 P-t-P:166.50.32.1 Mask:255.255.255.255 > UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1 > RX packets:4 errors:2 dropped:0 overruns:0 frame:0 > TX packets:7 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 > RX bytes:40 (40.0 b) TX bytes:92 (92.0 b) > > since there is no broadcast or network address that i can tell. > > what do i put for these variables..Unfortunately, this isn''t such an easy topic that you can just plug and play, specially when your situation is so much different then mine was when I wrote it. I mean, you''ll have to bother about the setting of the variables only when it comes to use them. For instance, in iptables, you wouldn'' use the SNAT but rather the MASQUERADE target, which doesn''t require any IP. On the other hand, if you have an dynamic IP, you will also have to set the routings as soon as the IP is known, i.e., yafter ou have connected and the provider told you what''s your IP for now. PPP doesn''t imply necessarily a dynamic IP, it''s just point-to-point, so I''m not really sure if all this still makes sense for you, because you''ll have to reconfigure your network anyway as soon as the connection drops. But if you have a static IP, you do know all of those things. Also, be aware that the use of both lines requires a big amount of traffic, which I was not able to generate alone. I just can''t imagine how you are going to route heavy traffic through a dialup line. -- Christoph Simon ciccio@kiosknet.com.br --- ^X^C q quit :q ^C end x exit ZZ ^D ? help .
On Tue, 18 Dec 2001, Christoph Simon wrote:> On Tue, 18 Dec 2001 14:27:20 -0500 (EST) > mok2 <mok2@armageddonnet.com> wrote: > > > Destination Gateway Genmask Flags Metric Ref Use Iface > > 166.50.32.1 0.0.0.0 255.255.255.255 UH 0 0 0 ppp0 > > 176.30.3.192 0.0.0.0 255.255.255.192 U 0 0 0 eth2 > > 10.2.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 > > 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo > > 0.0.0.0 176.30.3.193 0.0.0.0 UG 0 0 0 eth2 > > > > eth2 Link encap:Ethernet HWaddr > > inet addr:176.30.3.38 Bcast:176.30.3.255 Mask:255.255.255.192 > > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > > RX packets:98373 errors:0 dropped:0 overruns:0 frame:0 > > TX packets:116675 errors:0 dropped:0 overruns:0 carrier:0 > > collisions:1015 > > RX bytes:76086367 (72.5 Mb) TX bytes:97127829 (92.6 Mb) > > > > ppp0 Link encap:Point-to-Point Protocol > > inet addr:166.50.33.32 P-t-P:166.50.32.1 Mask:255.255.255.255 > > UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1 > > RX packets:4 errors:1 dropped:0 overruns:0 frame:0 > > TX packets:5 errors:0 dropped:0 overruns:0 carrier:0 > > collisions:0 > > RX bytes:40 (40.0 b) TX bytes:61 (61.0 b) > > > > the gw with the lowest metric determines which connection will work.. > > i cannot add in both gw and make both connections work at the same time.. > > > > that is my problem.. > > If you try to access 166.50.32.1/32, ppp0 will be used. If you try to > access 10.2.2.0/24, eth0 will be used. In all other cases, eth2 will > be used. This should just work. What do you expect to work and > doesn''t? BTW, normally there is no explicit route for lo; the kernel > does know how to deal with the loop back. >i cannot access 166.50.33.32 from the outside
On Tue, 18 Dec 2001 14:00:06 -0500 (EST) mok2 <mok2@armageddonnet.com> wrote:> > I am not tyring to route outbound traffice through ppp0 just recieve > traffic on that interface. > while still being connected to my cable provider.. > > and yes i have a dynamic ip on ppp0 and a static for my cable..In that case, I''m not sure to understand your problem. You are not going to have any multipath route; it a standard setup. -- Christoph Simon ciccio@kiosknet.com.br --- ^X^C q quit :q ^C end x exit ZZ ^D ? help .
if i change my default gw to the dialup isp than it works... but the cable gw now has a higher metric .. so it wont work now.. i can only use one of the connections at a time .. and the only one that works with is the connection with the lowest metric.. On Tue, 18 Dec 2001, Christoph Simon wrote:> On Tue, 18 Dec 2001 15:06:07 -0500 (EST) > mok2 <mok2@armageddonnet.com> wrote: > > > i cannot access 166.50.33.32 from the outside > > If you can access it from inside, the problem is at the other end. > > > -- > Christoph Simon > ciccio@kiosknet.com.br > --- > ^X^C > q > quit > :q > ^C > end > x > exit > ZZ > ^D > ? > help > . > > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/ >
ok its working now... thx.. On Tue, 18 Dec 2001, mok2 wrote:> > if i change my default gw to the dialup isp > than it works... > but the cable gw now has a higher metric .. so it wont work now.. > > i can only use one of the connections at a time .. and the only one that > works with is the connection with the lowest metric.. > > > On Tue, 18 Dec 2001, Christoph Simon wrote: > > > On Tue, 18 Dec 2001 15:06:07 -0500 (EST) > > mok2 <mok2@armageddonnet.com> wrote: > > > > > i cannot access 166.50.33.32 from the outside > > > > If you can access it from inside, the problem is at the other end. > > > > > > -- > > Christoph Simon > > ciccio@kiosknet.com.br > > --- > > ^X^C > > q > > quit > > :q > > ^C > > end > > x > > exit > > ZZ > > ^D > > ? > > help > > . > > > > _______________________________________________ > > LARTC mailing list / LARTC@mailman.ds9a.nl > > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/ > > > > > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/ >
On Tue, 18 Dec 2001 14:27:20 -0500 (EST) mok2 <mok2@armageddonnet.com> wrote:> Destination Gateway Genmask Flags Metric Ref Use Iface > 166.50.32.1 0.0.0.0 255.255.255.255 UH 0 0 0 ppp0 > 176.30.3.192 0.0.0.0 255.255.255.192 U 0 0 0 eth2 > 10.2.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 > 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo > 0.0.0.0 176.30.3.193 0.0.0.0 UG 0 0 0 eth2 > > eth2 Link encap:Ethernet HWaddr > inet addr:176.30.3.38 Bcast:176.30.3.255 Mask:255.255.255.192 > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:98373 errors:0 dropped:0 overruns:0 frame:0 > TX packets:116675 errors:0 dropped:0 overruns:0 carrier:0 > collisions:1015 > RX bytes:76086367 (72.5 Mb) TX bytes:97127829 (92.6 Mb) > > ppp0 Link encap:Point-to-Point Protocol > inet addr:166.50.33.32 P-t-P:166.50.32.1 Mask:255.255.255.255 > UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1 > RX packets:4 errors:1 dropped:0 overruns:0 frame:0 > TX packets:5 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 > RX bytes:40 (40.0 b) TX bytes:61 (61.0 b) > > the gw with the lowest metric determines which connection will work.. > i cannot add in both gw and make both connections work at the same time.. > > that is my problem..If you try to access 166.50.32.1/32, ppp0 will be used. If you try to access 10.2.2.0/24, eth0 will be used. In all other cases, eth2 will be used. This should just work. What do you expect to work and doesn''t? BTW, normally there is no explicit route for lo; the kernel does know how to deal with the loop back. -- Christoph Simon ciccio@kiosknet.com.br --- ^X^C q quit :q ^C end x exit ZZ ^D ? help .
On Tue, 18 Dec 2001 15:06:07 -0500 (EST) mok2 <mok2@armageddonnet.com> wrote:> i cannot access 166.50.33.32 from the outsideIf you can access it from inside, the problem is at the other end. -- Christoph Simon ciccio@kiosknet.com.br --- ^X^C q quit :q ^C end x exit ZZ ^D ? help .
On Tuesday 18 December 2001 16:03, bert hubert wrote:> > If we are using TBF, we can reduce "buffer" and "limit" to minimize > > latency. What should we do in SFQ? > > You can''t really. SFQ has a fixed queue length of 128. >Dear Sirs, It worked now. Thank you for all help, from this mailing list and from IRC. Special Thanks for Mr. Hubert I have reduce the buffer (queue length) from 128 to 15, and it works very nice, very well, and very fair. What I did was: 1. Open the sfq source (/usr/src/linux/net/sched/sch_sfq.c) 2. In about line 91, there is "#define SFQ_DEPTH 128" 3. Change it to "#define SFQ_DEPTH 15" 4. Recompile kernel (modules) 5. Restart (I''m not sure it is needed. modprobe should be enough) 6. Run it again 7. Nice...nice....nice..... For Mr. Kuznetsov, if you don''t mind, I would like to suggest to give a chance to the user to change it manually in run time, like we can change quantum and perturb. (in the next release, of course) In developed countries, queue depth 128 is fair, since we are talking about high speed network. In my case, I just use 64 kbit bandwidth for every 4 Internet caffee, which used by about 15 computers each. Very slow speed.... need very small queue length. Regards, Adi Nugroho Who are very happy now :-)
Hello!> For Mr. Kuznetsov, if you don''t mind, I would like to suggest to give a > chance to the user to change it manually in run time, like we can change > quantum and perturb. (in the next release, of course)Yes, of course. Actually, I do not remember why I did not make it tunable. :-) Seems, I was stuck on expenses of _increasing_ it and somewhat missed that people may want to decrease it as well. Alexey
On Wednesday 09 January 2002 11:30, Alexey Kuznetsov wrote:> > For Mr. Kuznetsov, if you don''t mind, I would like to suggest to give a > > chance to the user to change it manually in run time, like we can change > > quantum and perturb. (in the next release, of course) > > Yes, of course. > > AlexeyHurray...... Thank you for your help and your hardwork. Once the new sch_sfq.c has been updated, please don''t forget me.... I''m appliying to be a user tester :-) Once again.... thanks.... Adi Nugroho