LARTC - Dec 2007 - HTB performance improvement

HTB performance improvement 

Hi all !

i''m looking at the performance of the HTB algorithm/implementation
because i would like more packets/sec !!

this is the scenario of the performance test:

i''m using an embedded system with:

SPEED CPU: 399,999 MHz

RAM: 128 MByte

FLASH: 16 MByte

EEPROM: 8Kbyte

PROCESSOR MPC8272

a lan to lan 10/100 and in particular we are sending packets from a smartbits
(packet generator) to the embedded system above in the interface eth0, 10.0.0.2
-> passing trough QOS with HTB (embedded system)-> sending back
traffic through the interface eth1 which is attached to the smartbits on another
interface to measure received packets.

with packet length = 64 bytes @ layer 2, burst 120 sec, udp traffic, the test
i''m reporting are the upper limit (of packets/sec) with almost no drops
of packets, increasing packets/sec the embedded system/kernel algorithm starts
to drop packets.

1) with none qos configuration (system conf) i''ve obtained:  TX    
29940 packets/sec RX 29948.98 packets/sec with 2 packets lost

2) with only system conf + iptables marking support          TX/RX  24950
packets/sec, 8 packets lost

3) with only system conf + u32 support                       TX/RX  29950
packets/sec, 5 packets lost

4) with only system conf + HTB with iptables mark classifier TX/RX  13000
packets/sec, 0 packets lost

5) with only system conf + HTB with u32 classifier           TX/RX  19290
packets/sec, 0 packets lost

First of all, do you consider this scenario correct?

consider that i will not have more than 2/3 levels of classes, i''m
wondering if i can improve the algorithm with a better tc configuration (below
i''ve reported the exact configuration of the examples 4 and 5) or maybe
customize the algorithm for my particular use? do you think that part of the
performance responsibility is to be assigned at the linux QOS scheduler?

4) with only system conf + HTB with marking (iptables mark) classifier 13000
packets/sec (TX/RX), 0 packets lost:

insmod ip_tables.ko

insmod iptable_mangle.ko

insmod ipt_MARK.ko

insmod ipt_mark.ko

insmod cls_fw.ko

insmod sch_htb.ko

insmod sch_prio.ko

iptables -t mangle -A PREROUTING -i eth0.1 -p udp --dport 1001 -j MARK
--set-mark 1

iptables -t mangle -A PREROUTING -i eth0.1 -p udp --dport 1001 -j RETURN

iptables -t mangle -A PREROUTING -i eth0.1 -p udp --dport 1002 -j MARK
--set-mark 2

iptables -t mangle -A PREROUTING -i eth0.1 -p udp --dport 1002 -j RETURN

iptables -t mangle -A PREROUTING -i eth0.1 -p udp --dport 1003 -j MARK
--set-mark 3

iptables -t mangle -A PREROUTING -i eth0.1 -p udp --dport 1003 -j RETURN

iptables -t mangle -A PREROUTING -j MARK --set-mark 4

tc qdisc add dev eth0.2 root handle 1: htb

tc class add dev eth0.2 parent 1: classid 1:1 htb  rate 100000000 ceil 100000000
burst 1000000 quantum 1600

tc class add dev eth0.2 parent 1:1 classid 1:10 htb prio 1 rate 70000000 ceil
70000000 burst 1000000 quantum 1600

tc class add dev eth0.2 parent 1:1 classid 1:11 htb prio 2 rate 10000000 ceil
10000000 burst 1000000 quantum 1600

tc class add dev eth0.2 parent 1:1 classid 1:12 htb prio 3 rate 10000000 ceil
10000000 burst 1000000 quantum 1600

tc filter add dev eth0.2 parent 1: protocol ip prio 1 handle 1 fw classid 1:10

tc filter add dev eth0.2 parent 1: protocol ip prio 2 handle 2 fw classid 1:11

tc filter add dev eth0.2 parent 1: protocol ip prio 3 handle 3 fw classid 1:12

5) with only system conf + HTB with u32 classifier 19290 packets/sec (TX/RX), 0
packets lost:

insmod sch_htb.ko

insmod sch_prio.ko

insmod cls_u32.ko

tc qdisc add dev eth0.2 root handle 1: htb default 12

tc class add dev eth0.2 parent 1: classid 1:1 htb  rate 100000000 ceil 100000000
burst 735000 quantum 1600

tc class add dev eth0.2 parent 1:1 classid 1:10 htb prio 1 rate 70000000 ceil
70000000 burst 324000 quantum 1600

tc class add dev eth0.2 parent 1:1 classid 1:11 htb prio 2 rate 10000000 ceil
10000000 burst 411000 quantum 1600

tc class add dev eth0.2 parent 1:1 classid 1:12 htb prio 3 rate 10000000 ceil
10000000 burst 411000 quantum 1600

tc filter add dev eth0.2 parent 1: protocol ip prio 1 u32 match ip dport 1001
0xffff classid 1:10

#filtro tos

#tc filter add dev eth0.2 parent 1: protocol ip prio 1 u32 match u32 0x00020000
0x00ff0000 at 0 classid 1:10

Thanks a lot !

Mingyur


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

> Koblensky Mingyur wrote:
> 
> Hi all !
> i''m looking at the performance of the HTB algorithm/implementation
> because i would like more packets/sec !!
> this is the scenario of the performance test:
> 
> i''m using an embedded system with:
> SPEED CPU: 399,999 MHz
> RAM: 128 MByte
> FLASH: 16 MByte
> EEPROM: 8Kbyte
> PROCESSOR MPC8272
> 
> 1) with none qos configuration (system conf) i''ve obtained:  TX
> 29940 packets/sec RX 29948.98 packets/sec with 2 packets lost
> 2) with only system conf + iptables marking support          TX/RX
> 24950 packets/sec, 8 packets lost
> 3) with only system conf + u32 support                       TX/RX
> 29950 packets/sec, 5 packets lost
> 4) with only system conf + HTB with iptables mark classifier TX/RX
> 13000 packets/sec, 0 packets lost
> 5) with only system conf + HTB with u32 classifier           TX/RX
> 19290 packets/sec, 0 packets lost
> 
> tc qdisc add dev eth0.2 root handle 1: htb
> tc class add dev eth0.2 parent 1: classid 1:1 htb  rate 100000000 ceil
> 100000000 burst 1000000 quantum 1600
> tc class add dev eth0.2 parent 1:1 classid 1:10 htb prio 1 rate
> 70000000 ceil 70000000 burst 1000000 quantum 1600
> tc class add dev eth0.2 parent 1:1 classid 1:11 htb prio 2 rate
> 10000000 ceil 10000000 burst 1000000 quantum 1600
> tc class add dev eth0.2 parent 1:1 classid 1:12 htb prio 3 rate
> 10000000 ceil 10000000 burst 1000000 quantum 1600

First, your quantum is wrong.  Think of HTB as a NIC.  You are sending
packets out via that "NIC", and those packets must be 1600 bytes big
before being sent.  Since a "normal" packet is up to 1500 bytes,
you''re
making HTB take part of the next packet before it can send.

Your burst is also huge enough to swamp the interrupt controller. 
Consider 16K or 32K instead.

Unless you get complaints during the load of the above, do not specify
any quantum, or specify the quantum HTB sets for rates that do not
complain.

The next likely reason for you to be at "half spped" is that HTB is
using the wrong clock.  Which clock and what speed are kernel version
dependent.

Part of what I snipped referred to 10/100 ethernet.  Normally that will
not trigger any, but be sure that when you run a ''tc -s class ls dev
$IFE'' (where IFE is the correct interface) that "giants" are
always
zero.
--
gypsy

gypsy wrote:
> 
> > tc qdisc add dev eth0.2 root handle 1: htb
> > tc class add dev eth0.2 parent 1: classid 1:1 htb  rate 100000000 ceil
> > 100000000 burst 1000000 quantum 1600
> > tc class add dev eth0.2 parent 1:1 classid 1:10 htb prio 1 rate
> > 70000000 ceil 70000000 burst 1000000 quantum 1600
> > tc class add dev eth0.2 parent 1:1 classid 1:11 htb prio 2 rate
> > 10000000 ceil 10000000 burst 1000000 quantum 1600
> > tc class add dev eth0.2 parent 1:1 classid 1:12 htb prio 3 rate
> > 10000000 ceil 10000000 burst 1000000 quantum 1600
I just noticed that you''re trying to shape "aliases".  You
can''t.  You
must shape eth0 because you can''t shape eth0.2.  I''m just
repeating what
I''ve heard/read, so do your own research.  But I bet you a quarter
I''m
right :)
--
gypsy

gypsy wrote:
> gypsy wrote:
>>> tc qdisc add dev eth0.2 root handle 1: htb
>>> tc class add dev eth0.2 parent 1: classid 1:1 htb  rate 100000000
ceil
>>> 100000000 burst 1000000 quantum 1600
>>> tc class add dev eth0.2 parent 1:1 classid 1:10 htb prio 1 rate
>>> 70000000 ceil 70000000 burst 1000000 quantum 1600
>>> tc class add dev eth0.2 parent 1:1 classid 1:11 htb prio 2 rate
>>> 10000000 ceil 10000000 burst 1000000 quantum 1600
>>> tc class add dev eth0.2 parent 1:1 classid 1:12 htb prio 3 rate
>>> 10000000 ceil 10000000 burst 1000000 quantum 1600
> 
> I just noticed that you''re trying to shape "aliases". 
You can''t.  You
> must shape eth0 because you can''t shape eth0.2.  I''m just
repeating what
> I''ve heard/read, so do your own research.  But I bet you a quarter
I''m
> right :)
> --
Could be vlans - in which case OP could try adding child b/pfifos to the 
leafs and use the limit parameter to specify buffer length.

HTB and HFSC both pick up default queuelengths from the device, but
it''s
0 on vlans (they use 2 or 3 packet pfifos IIRC in this case).

I do notice there are no drops in the stats though, but it''s still
worth
specifying queue length and type in the script.

Andy.