Hi! This morning one of our major links died and suddenly I was streaming 1mbit of traffic through 256kbit (one of our backup links), but it just doesn't work. The client is still doing in the region of 350kbit/s (the link capacity is 512kbit). Here is my config: ------------< snip <------< snip <------< snip <------------ tc qdisc add dev eth1 root handle 1: htb default 4 r2q 5 tc class add dev eth1 parent 1: classid 1:1 htb rate 512kbit tc class add dev eth1 parent 1:1 classid 1:2 htb rate 320kbit ceil 512kbit prio 1 tc qdisc add dev eth1 handle 2: parent 1:2 sfq perturb 10 limit 84 tc class add dev eth1 parent 1:1 classid 1:3 htb rate 64kbit ceil 256kbit prio 3 tc qdisc add dev eth1 handle 3: parent 1:3 sfq perturb 10 limit 42 tc class add dev eth1 parent 1:1 classid 1:4 htb rate 128kbit ceil 512kbit prio 2 tc qdisc add dev eth1 handle 4: parent 1:4 sfq perturb 10 limit 84 tc qdisc add dev eth0 root handle 1: htb default 4 r2q 5 tc class add dev eth0 parent 1: classid 1:1 htb rate 512kbit tc class add dev eth0 parent 1:1 classid 1:2 htb rate 320kbit ceil 512kbit prio 1 tc qdisc add dev eth0 handle 2: parent 1:2 sfq perturb 10 limit 84 tc class add dev eth0 parent 1:1 classid 1:3 htb rate 64kbit ceil 256kbit prio 3 tc qdisc add dev eth0 handle 3: parent 1:3 sfq perturb 10 limit 42 tc class add dev eth0 parent 1:1 classid 1:4 htb rate 128kbit ceil 512kbit prio 2 tc qdisc add dev eth0 handle 4: parent 1:4 sfq perturb 10 limit 84 ------------< snip <------< snip <------< snip <------------ I'm tagging packets with the netfilter CLASSIFY patch. I've got a user-defined chain in the mangle table and I redirect all traffic from the POSTROUTING chain into that and add rules like this: iptables -t mangle -N qos iptables -t mangle -A POSTROUTING -j qos and then these rules for the ftc rules above: ------------< snip <------< snip <------< snip <------------ iptables -t mangle -A qos -o eth1 -d 66.8.28.92/30 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth1 -d 66.8.28.160/27 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth1 -d 168.210.55.0/24 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth1 -d 168.210.54.80/29 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth1 -d 168.210.56.0/24 -j CLASSIFY --set-class 1:3 iptables -t mangle -A qos -o eth1 -d 66.8.28.32/27 -j CLASSIFY --set-class 1:3 iptables -t mangle -A qos -o eth0 -s 66.8.28.92/30 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth0 -s 66.8.28.160/27 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth0 -s 168.210.55.0/24 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth0 -s 168.210.54.80/29 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth0 -s 168.210.56.0/24 -j CLASSIFY --set-class 1:3 iptables -t mangle -A qos -o eth0 -s 66.8.28.32/27 -j CLASSIFY --set-class 1:3 ------------< snip <------< snip <------< snip <------------ The class which is not shaping correctly is 1:3 - it has a rate of 64kbit and a ceil of 256kbit so it should be limited to 256kbit even if there is 512kbit of traffic available. Any ideas what I did wrong or is this a bug? -- Regards Abraham Bernard Shaw is an excellent man; he has not an enemy in the world, and none of his friends like him either. -- Oscar Wilde ___________________________________________________ Abraham vd Merwe - Frogfoot Networks CC 9 Kinnaird Court, 33 Main Street, Newlands, 7700 Phone: +27 21 686 1665 Cell: +27 82 565 4451 Http: http://www.frogfoot.net/ Email: abz@frogfoot.net
On Sun, 21 Sep 2003, Abraham van der Merwe wrote: I am not sure, but I think if you classify your packet in PREROUTING chain it will works fine > Hi! > > This morning one of our major links died and suddenly I was streaming 1mbit > of traffic through 256kbit (one of our backup links), but it just doesn't > work. The client is still doing in the region of 350kbit/s (the link > capacity is 512kbit). Here is my config: > > ------------< snip <------< snip <------< snip <------------ > tc qdisc add dev eth1 root handle 1: htb default 4 r2q 5 > tc class add dev eth1 parent 1: classid 1:1 htb rate 512kbit > tc class add dev eth1 parent 1:1 classid 1:2 htb rate 320kbit ceil 512kbit prio 1 > tc qdisc add dev eth1 handle 2: parent 1:2 sfq perturb 10 limit 84 > tc class add dev eth1 parent 1:1 classid 1:3 htb rate 64kbit ceil 256kbit prio 3 > tc qdisc add dev eth1 handle 3: parent 1:3 sfq perturb 10 limit 42 > tc class add dev eth1 parent 1:1 classid 1:4 htb rate 128kbit ceil 512kbit prio 2 > tc qdisc add dev eth1 handle 4: parent 1:4 sfq perturb 10 limit 84 > tc qdisc add dev eth0 root handle 1: htb default 4 r2q 5 > tc class add dev eth0 parent 1: classid 1:1 htb rate 512kbit > tc class add dev eth0 parent 1:1 classid 1:2 htb rate 320kbit ceil 512kbit prio 1 > tc qdisc add dev eth0 handle 2: parent 1:2 sfq perturb 10 limit 84 > tc class add dev eth0 parent 1:1 classid 1:3 htb rate 64kbit ceil 256kbit prio 3 > tc qdisc add dev eth0 handle 3: parent 1:3 sfq perturb 10 limit 42 > tc class add dev eth0 parent 1:1 classid 1:4 htb rate 128kbit ceil 512kbit prio 2 > tc qdisc add dev eth0 handle 4: parent 1:4 sfq perturb 10 limit 84 > ------------< snip <------< snip <------< snip <------------ > > I'm tagging packets with the netfilter CLASSIFY patch. I've got a > user-defined chain in the mangle table and I redirect all traffic from the > POSTROUTING chain into that and add rules like this: > > iptables -t mangle -N qos > iptables -t mangle -A POSTROUTING -j qos > > and then these rules for the ftc rules above: > > ------------< snip <------< snip <------< snip <------------ > iptables -t mangle -A qos -o eth1 -d 66.8.28.92/30 -j CLASSIFY --set-class 1:2 > iptables -t mangle -A qos -o eth1 -d 66.8.28.160/27 -j CLASSIFY --set-class 1:2 > iptables -t mangle -A qos -o eth1 -d 168.210.55.0/24 -j CLASSIFY --set-class 1:2 > iptables -t mangle -A qos -o eth1 -d 168.210.54.80/29 -j CLASSIFY --set-class 1:2 > iptables -t mangle -A qos -o eth1 -d 168.210.56.0/24 -j CLASSIFY --set-class 1:3 > iptables -t mangle -A qos -o eth1 -d 66.8.28.32/27 -j CLASSIFY --set-class 1:3 > iptables -t mangle -A qos -o eth0 -s 66.8.28.92/30 -j CLASSIFY --set-class 1:2 > iptables -t mangle -A qos -o eth0 -s 66.8.28.160/27 -j CLASSIFY --set-class 1:2 > iptables -t mangle -A qos -o eth0 -s 168.210.55.0/24 -j CLASSIFY --set-class 1:2 > iptables -t mangle -A qos -o eth0 -s 168.210.54.80/29 -j CLASSIFY --set-class 1:2 > iptables -t mangle -A qos -o eth0 -s 168.210.56.0/24 -j CLASSIFY --set-class 1:3 > iptables -t mangle -A qos -o eth0 -s 66.8.28.32/27 -j CLASSIFY --set-class 1:3 > ------------< snip <------< snip <------< snip <------------ > > The class which is not shaping correctly is 1:3 - it has a rate of 64kbit > and a ceil of 256kbit so it should be limited to 256kbit even if there is > 512kbit of traffic available. > > Any ideas what I did wrong or is this a bug? > >
Hi Mahmoud >@2003.09.22_11:44:01_+0200
(a) You can't put it in PREROUTING - the CLASSIFY patch don't allow that:
from net/ipv4/netfilter/ipt_CLASSIFY.c:
------------< snip <------< snip <------< snip <------------
if (hook_mask & ~(1 << NF_IP_POST_ROUTING)) {
printk(KERN_ERR "CLASSIFY: only valid in POST_ROUTING.\n");
return 0;
}
------------< snip <------< snip <------< snip <------------
Would you mind telling me why you suggested PREROUTING?
(b)
I discovered some serious problems in my ruleset below which I fixed in the
meantime (Notice all the qdiscs have the same major id - they're supposed to
be unique for each qdisc, i.e. the sfq qdisc's and the two htb qdisc's
should have different major numbers)
(c)
One thing I discovered as well is that if all traffic goes through one of
our backup links, the line is so congested that I still get around double
the traffic incoming on the qos box itself. If I check on a box behind it
you can see that that class is shaped correctly, the requests are just
flooding in at such a pace that it still eats up twice the bandwidth!
> I am not sure, but I think if you classify your packet in PREROUTING chain
> it will works fine
>
> > Hi!
> >
> > This morning one of our major links died and suddenly I was streaming
1mbit
> > of traffic through 256kbit (one of our backup links), but it just
doesn't
> > work. The client is still doing in the region of 350kbit/s (the link
> > capacity is 512kbit). Here is my config:
> >
> > ------------< snip <------< snip <------< snip
<------------
> > tc qdisc add dev eth1 root handle 1: htb default 4 r2q 5
> > tc class add dev eth1 parent 1: classid 1:1 htb rate 512kbit
> > tc class add dev eth1 parent 1:1 classid 1:2 htb rate 320kbit ceil
512kbit prio 1
> > tc qdisc add dev eth1 handle 2: parent 1:2 sfq perturb 10 limit 84
> > tc class add dev eth1 parent 1:1 classid 1:3 htb rate 64kbit ceil
256kbit prio 3
> > tc qdisc add dev eth1 handle 3: parent 1:3 sfq perturb 10 limit 42
> > tc class add dev eth1 parent 1:1 classid 1:4 htb rate 128kbit ceil
512kbit prio 2
> > tc qdisc add dev eth1 handle 4: parent 1:4 sfq perturb 10 limit 84
> > tc qdisc add dev eth0 root handle 1: htb default 4 r2q 5
> > tc class add dev eth0 parent 1: classid 1:1 htb rate 512kbit
> > tc class add dev eth0 parent 1:1 classid 1:2 htb rate 320kbit ceil
512kbit prio 1
> > tc qdisc add dev eth0 handle 2: parent 1:2 sfq perturb 10 limit 84
> > tc class add dev eth0 parent 1:1 classid 1:3 htb rate 64kbit ceil
256kbit prio 3
> > tc qdisc add dev eth0 handle 3: parent 1:3 sfq perturb 10 limit 42
> > tc class add dev eth0 parent 1:1 classid 1:4 htb rate 128kbit ceil
512kbit prio 2
> > tc qdisc add dev eth0 handle 4: parent 1:4 sfq perturb 10 limit 84
> > ------------< snip <------< snip <------< snip
<------------
> >
> > I'm tagging packets with the netfilter CLASSIFY patch. I've got a
> > user-defined chain in the mangle table and I redirect all traffic from
the
> > POSTROUTING chain into that and add rules like this:
> >
> > iptables -t mangle -N qos
> > iptables -t mangle -A POSTROUTING -j qos
> >
> > and then these rules for the ftc rules above:
> >
> > ------------< snip <------< snip <------< snip
<------------
> > iptables -t mangle -A qos -o eth1 -d 66.8.28.92/30 -j CLASSIFY
--set-class 1:2
> > iptables -t mangle -A qos -o eth1 -d 66.8.28.160/27 -j CLASSIFY
--set-class 1:2
> > iptables -t mangle -A qos -o eth1 -d 168.210.55.0/24 -j CLASSIFY
--set-class 1:2
> > iptables -t mangle -A qos -o eth1 -d 168.210.54.80/29 -j CLASSIFY
--set-class 1:2
> > iptables -t mangle -A qos -o eth1 -d 168.210.56.0/24 -j CLASSIFY
--set-class 1:3
> > iptables -t mangle -A qos -o eth1 -d 66.8.28.32/27 -j CLASSIFY
--set-class 1:3
> > iptables -t mangle -A qos -o eth0 -s 66.8.28.92/30 -j CLASSIFY
--set-class 1:2
> > iptables -t mangle -A qos -o eth0 -s 66.8.28.160/27 -j CLASSIFY
--set-class 1:2
> > iptables -t mangle -A qos -o eth0 -s 168.210.55.0/24 -j CLASSIFY
--set-class 1:2
> > iptables -t mangle -A qos -o eth0 -s 168.210.54.80/29 -j CLASSIFY
--set-class 1:2
> > iptables -t mangle -A qos -o eth0 -s 168.210.56.0/24 -j CLASSIFY
--set-class 1:3
> > iptables -t mangle -A qos -o eth0 -s 66.8.28.32/27 -j CLASSIFY
--set-class 1:3
> > ------------< snip <------< snip <------< snip
<------------
> >
> > The class which is not shaping correctly is 1:3 - it has a rate of
64kbit
> > and a ceil of 256kbit so it should be limited to 256kbit even if there
is
> > 512kbit of traffic available.
> >
> > Any ideas what I did wrong or is this a bug?
> >
> >
>
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
>
--
Regards
Abraham
But Officer, I stopped for the last one, and it was green!
___________________________________________________
Abraham vd Merwe - Frogfoot Networks CC
9 Kinnaird Court, 33 Main Street, Newlands, 7700
Phone: +27 21 686 1665 Cell: +27 82 565 4451
Http: http://www.frogfoot.net/ Email: abz@frogfoot.net