LARTC - Sep 2006 - how to setup massive traffic shaping? (2 class B nets)

Hello
I have 2 class-B networks (172.22.0.0/16 and 172.23.0.0/16, over 130k
of ip''s) and need to setup
traffic tbf shapers  with 64kb/s for each ip from 172.22.0.0/16 and
128kb/s for each ip from 172.23.0.0/16
just read lartc and don''t understand how to use u32 for decreasing
number of rules and hashing

????? ????? wrote:
> Hello
> I have 2 class-B networks (172.22.0.0/16 and 172.23.0.0/16, over 130k
> of ip''s) and need to setup
> traffic tbf shapers with 64kb/s for each ip from 172.22.0.0/16 and
> 128kb/s for each ip from 172.23.0.0/16
> just read lartc and don''t understand how to use u32 for decreasing
> number of rules and hashing
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
First you need imq. This because you can''t have more than 65536
classes.
Having 8 imqs you will be able to use 4 for download and 4 for upload. 
However You will tell us if the machine will support everything :). So 
you will divide the big class /15 into 4 classes /17

iptables -t mangle -A POSTROUTING -o $clients_eth -d 172.22.0.0/17 -j 
IMQ --todev 0

and:

tc qdisc add dev imq0 root handle 1: htb #Or whatever you use
tc class add dev imq0 parent 1: classid 1:f000 htb rate 620mbit #or 
whatever ...


tc filter add dev imq0 parent 1: protocol ip prio 5 handle 0: u32 
divisor 128
tc filter add dev imq0 parent 1: prio 5 u32 ht 800:: match ip dst 
172.22.0.0/17 hashkey mask 0x00007f00 at 16 link 0:

for i in seq `0 127` ; do
div=`printf "%x" $i`
let j=$i+1
tc filter add dev imq0 parent 1: prio 5 u32 ht 0:$div: match ip dst 
172.22.$i.0/24 hashkey mask 0x000000ff at 16 link $j:
for q in seq `0 255`; do
d=`printf "%x" $q`
clsid=$[$i*256+$q]
tc class add dev imq0 parent 1: classid 1:$clsid htb rate 128kbit
tc filter add dev imq0 parent 1: prio 5 u32 ht $j:$d match ip dst 
172.22.$i.$q flowid 1:$clsid
done
done

I hope i''m not missing something. For 172.22.128.0/17, instead of 
172.22.$i you will have 172.22.$[$i+128]
On other imq of course. For upload you just change dst with src, and "at 
16" into "at 12"


I''m really curious how it will work :D and how much it will take to run
it.

Instead of
tc class add dev imq0 parent 1: classid 1:$clsid htb rate 128kbit

Use

tc class add dev imq0 parent 1:f000 classid 1:$clsid htb rate 128kbit

????? ????? wrote:
> Hello!
> yes, I no about 65000 rules and just add third interface to server. )
> what about
> tc filter add dev imq0 parent 1: prio 5 u32 ht 800:: match ip dst
> 172.22.0.0/17 hashkey mask 0x00007f00 at 16 link 0: ?
> shouldnt hashkey mask be 0x00007fff instead of 0x00007f00 ?
> and if it should be 0x00007f00 how do you calculate it?
>
>
> 2006/9/20, Alexandru Dragoi <alex@zoomnet.ro>:
>> ????? ????? wrote:
>> > Hello
>> > I have 2 class-B networks (172.22.0.0/16 and 172.23.0.0/16, over
130k
>> > of ip''s) and need to setup
>> > traffic tbf shapers with 64kb/s for each ip from 172.22.0.0/16 and
>> > 128kb/s for each ip from 172.23.0.0/16
>> > just read lartc and don''t understand how to use u32 for
decreasing
>> > number of rules and hashing
>> > _______________________________________________
>> > LARTC mailing list
>> > LARTC@mailman.ds9a.nl
>> > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>> First you need imq. This because you can''t have more than
65536 classes.
>> Having 8 imqs you will be able to use 4 for download and 4 for upload.
>> However You will tell us if the machine will support everything :). So
>> you will divide the big class /15 into 4 classes /17
>>
>> iptables -t mangle -A POSTROUTING -o $clients_eth -d 172.22.0.0/17 -j
>> IMQ --todev 0
>>
>> and:
>>
>> tc qdisc add dev imq0 root handle 1: htb #Or whatever you use
>> tc class add dev imq0 parent 1: classid 1:f000 htb rate 620mbit #or
>> whatever ...
>>
>>
>> tc filter add dev imq0 parent 1: protocol ip prio 5 handle 0: u32
>> divisor 128
>> tc filter add dev imq0 parent 1: prio 5 u32 ht 800:: match ip dst
>> 172.22.0.0/17 hashkey mask 0x00007f00 at 16 link 0:
>>
>> for i in seq `0 127` ; do
>> div=`printf "%x" $i`
>> let j=$i+1
>> tc filter add dev imq0 parent 1: prio 5 u32 ht 0:$div: match ip dst
>> 172.22.$i.0/24 hashkey mask 0x000000ff at 16 link $j:
>> for q in seq `0 255`; do
>> d=`printf "%x" $q`
>> clsid=$[$i*256+$q]
>> tc class add dev imq0 parent 1: classid 1:$clsid htb rate 128kbit
>> tc filter add dev imq0 parent 1: prio 5 u32 ht $j:$d match ip dst
>> 172.22.$i.$q flowid 1:$clsid
>> done
>> done
>>
>> I hope i''m not missing something. For 172.22.128.0/17, instead
of
>> 172.22.$i you will have 172.22.$[$i+128]
>> On other imq of course. For upload you just change dst with src, and
"at
>> 16" into "at 12"
>>
>>
>> I''m really curious how it will work :D and how much it will
take to
>> run it.
>>
>>
>>
Also, before:

tc filter add dev imq0 parent 1: prio 5 u32 ht 0:$div: match ip dst 
172.22.$i.0/24 hashkey mask 0x000000ff at 16 link $j:

Add line:

tc filter add dev imq0 parent 1: protocol ip prio 5 handle $j: u32 
divisor 256

Now, you need both download and upload, so maybe upload goes via a 
single card. Also the maximum number of divisor is 256, so a mask should 
have only 8 bits set to 1. And 0x00007f00 will match only the last 7 
bits on 3rd number on an ip address. So ip 172.22.45.2 will go, 
considering the hashkey mask, to entry (45 & 0x7f), then represented in 
hex, which is 0x23. The & means bitwise and, just how hashkey algoritm 
works. Applying the that mask to ip 172.22.173.8 will also generate 
0x23. More about these you have to do some binary operations. Means you 
will run the for loop for i in `seq 128 255`.

Hello, Alexandru
I just adopt your script for B class net
is it correct?

tc qdisc add dev eth0 root handle 1: htb
tc class add dev eth0 parent 1: classid 1:f000 htb rate 620mbit


tc filter add dev eth0 parent 1: protocol ip prio 5 handle 0: u32 divisor 256
tc filter add dev eth0 parent 1: prio 5 u32 ht 800:: match ip dst
172.26.0.0/16 hashkey mask 0x0000ff00 at 16 link 0:



for i in `seq 0 255` ; do
div=`printf "%x" $i`
let j=$i+1
tc filter add dev eth0 parent 1: protocol ip prio 5 handle $j: u32 divisor 256
tc filter add dev eth0 parent 1: prio 5 u32 ht 0:$div: match ip dst
172.26.$i.0/24 hashkey mask 0x000000ff at 16 link $j:

for q in `seq 0 255` ; do
d=`printf "%x" $q`
clsid=$[$i*256+$q]
tc class add dev eth0 parent 1:f000 classid 1:$clsid htb rate 64kbit
tc filter add dev eth0 parent 1: prio 5 u32 ht $j:$d match ip dst
172.26.$i.$q flowid 1:$clsid
done
done


2006/9/20, Alexandru Dragoi <alex@zoomnet.ro>:
> ????? ????? wrote:
> > Hello!
> > yes, I no about 65000 rules and just add third interface to server. )
> > what about
> > tc filter add dev imq0 parent 1: prio 5 u32 ht 800:: match ip dst
> > 172.22.0.0/17 hashkey mask 0x00007f00 at 16 link 0: ?
> > shouldnt hashkey mask be 0x00007fff instead of 0x00007f00 ?
> > and if it should be 0x00007f00 how do you calculate it?
> >
> >
> > 2006/9/20, Alexandru Dragoi <alex@zoomnet.ro>:
> >> ????? ????? wrote:
> >> > Hello
> >> > I have 2 class-B networks (172.22.0.0/16 and 172.23.0.0/16,
over 130k
> >> > of ip''s) and need to setup
> >> > traffic tbf shapers with 64kb/s for each ip from
172.22.0.0/16 and
> >> > 128kb/s for each ip from 172.23.0.0/16
> >> > just read lartc and don''t understand how to use u32
for decreasing
> >> > number of rules and hashing
> >> > _______________________________________________
> >> > LARTC mailing list
> >> > LARTC@mailman.ds9a.nl
> >> > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
> >> First you need imq. This because you can''t have more than
65536 classes.
> >> Having 8 imqs you will be able to use 4 for download and 4 for
upload.
> >> However You will tell us if the machine will support everything
:). So
> >> you will divide the big class /15 into 4 classes /17
> >>
> >> iptables -t mangle -A POSTROUTING -o $clients_eth -d 172.22.0.0/17
-j
> >> IMQ --todev 0
> >>
> >> and:
> >>
> >> tc qdisc add dev imq0 root handle 1: htb #Or whatever you use
> >> tc class add dev imq0 parent 1: classid 1:f000 htb rate 620mbit
#or
> >> whatever ...
> >>
> >>
> >> tc filter add dev imq0 parent 1: protocol ip prio 5 handle 0: u32
> >> divisor 128
> >> tc filter add dev imq0 parent 1: prio 5 u32 ht 800:: match ip dst
> >> 172.22.0.0/17 hashkey mask 0x00007f00 at 16 link 0:
> >>
> >> for i in seq `0 127` ; do
> >> div=`printf "%x" $i`
> >> let j=$i+1
> >> tc filter add dev imq0 parent 1: prio 5 u32 ht 0:$div: match ip
dst
> >> 172.22.$i.0/24 hashkey mask 0x000000ff at 16 link $j:
> >> for q in seq `0 255`; do
> >> d=`printf "%x" $q`
> >> clsid=$[$i*256+$q]
> >> tc class add dev imq0 parent 1: classid 1:$clsid htb rate 128kbit
> >> tc filter add dev imq0 parent 1: prio 5 u32 ht $j:$d match ip dst
> >> 172.22.$i.$q flowid 1:$clsid
> >> done
> >> done
> >>
> >> I hope i''m not missing something. For 172.22.128.0/17,
instead of
> >> 172.22.$i you will have 172.22.$[$i+128]
> >> On other imq of course. For upload you just change dst with src,
and "at
> >> 16" into "at 12"
> >>
> >>
> >> I''m really curious how it will work :D and how much it
will take to
> >> run it.
> >>
> >>
> >>
> Also, before:
>
> tc filter add dev imq0 parent 1: prio 5 u32 ht 0:$div: match ip dst
> 172.22.$i.0/24 hashkey mask 0x000000ff at 16 link $j:
>
> Add line:
>
> tc filter add dev imq0 parent 1: protocol ip prio 5 handle $j: u32
> divisor 256
>
> Now, you need both download and upload, so maybe upload goes via a
> single card. Also the maximum number of divisor is 256, so a mask should
> have only 8 bits set to 1. And 0x00007f00 will match only the last 7
> bits on 3rd number on an ip address. So ip 172.22.45.2 will go,
> considering the hashkey mask, to entry (45 & 0x7f), then represented in
> hex, which is 0x23. The & means bitwise and, just how hashkey algoritm
> works. Applying the that mask to ip 172.22.173.8 will also generate
> 0x23. More about these you have to do some binary operations. Means you
> will run the for loop for i in `seq 128 255`.
>

If you use HTB, you need to compile it with HTB_HSIZE set to at least 256 (in 
sch_htb.c). Else your CPU will be fully loaded with even a few kpps traffic.

The problem is how HTB stores the classes, not very efficient when there are 
thousands of them. I do not know if other qdiscs have the same problem.

I am working on a better patch for that, but it is not ready yet.


Regards,
Simon


On Tuesday 19 September 2006 13:42, Тимур Сафин wrote:
> Hello
> I have 2 class-B networks (172.22.0.0/16 and 172.23.0.0/16, over 130k
> of ip''s) and need to setup
> traffic tbf shapers  with 64kb/s for each ip from 172.22.0.0/16 and
> 128kb/s for each ip from 172.23.0.0/16
> just read lartc and don''t understand how to use u32 for decreasing
> number of rules and hashing
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc