LARTC - Nov 2006 - Looking for new ideas to improve linux router performace

Hello,

I have 2 dual CPU Xeon 3Ghz HT enabled Linux routers and each one of them
serving 2 class C with pick traffic on router about 300Mbit full duplex

 

2 x Ethernet controller: Intel Corporation 82546GB Gigabit Ethernet
Controller (rev 03) 

1GB ram

And 

2 x Ethernet controller: Broadcom Corporation NetXtreme BCM5703 Gigabit
Ethernet (rev 10)

1GB ram

 

Both routers have about 600 iptables rules, 4000 tc rules with HFSC
scheduler and 300 static routes 

I have implemented tc filter hashing which improve performance 

but my goal is to push those machines to the limit with 4 class C and double
above rules and traffic

Currently system takes no more then 60% CPU time at pick per working CPU as
every NIC has been set on different CPU,

and I have two idle CPUs on each machine .

   

 Currently Linux kernels are coming with timer interrupt of 1000hz max which
in my opinion is not enough 

 Also there is no way to serve interrupts from one NIC on two processors.

 I''m open for suggestions 

 Thanks to all in advance.



_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

The reasons for higher CPU usage is that my iptables rules are linear
without them CPU usage drops from 60% to 30% pick and during day time it is
not more then 20% write now I don''t have time to implement ipset 
As for "Timer frequency" 250 no accurate traffic control at high
speeds can
be achieved. Also for 6 class C my bandwidth while be 900Mbit not 400Mbit.



-----Original Message-----
From: Konstantin Astafjev [mailto:konstantin@astafjev.com] 
Sent: Wednesday, November 08, 2006 3:16 PM
To: administrator@netwlan.net
Subject: Re: [LARTC] Looking for new ideas to improve linux router
performace

Hello administrator,

Wednesday, November 8, 2006, 2:39:08 PM, you wrote:
> I have 2 dual CPU Xeon 3Ghz HT enabled Linux routers and each one
> of them serving 2 class C with pick traffic on router about 300Mbit full
duplex

I have 1 common desktop PC router with Athlon64 1800MHz 1GB RAM
serving 6 C classes with one Intel server NIC at about 400Mbit load of
full duplex in average.
> Both routers have about 600 iptables rules, 4000 tc rules with HFSC
scheduler and 300 static routes

I have more then 1000 iptables rules (not linear, but in tree search
variant), about 3000 tc filter rules with HTB, only one default route.
And of course I''m using hash tables. BTW, is there any method to
classify fwmark in hash tables? Cause right now I have to use TOS for
that. :(
> I have implemented tc filter hashing which improve performance
>   
> but my goal is to push those machines to the limit with 4 class C
> and double above rules and traffic
I plan to increase quantity of clients to 8 C classes on this week.
> Currently system takes no more then 60% CPU time at pick per
> working CPU as every NIC has been set on different CPU,
CPU load of mine is about 45% at peak and I have only one CPU.
>  Currently Linux kernels are coming with timer interrupt of 1000hz
> max which in my opinion is not enough 
If you are about "Timer frequency" then mine is set to 250 HZ.
>  Also there is no way to serve interrupts from one NIC on two processors.
I have a 2 Opteron (4 cores) server right now with 2 tg3 NICs. I''ll
try to do an experiment to use it as a router. I''m also afraid that
the other 3 cores will not be used. :(

Very interesting numbers for me. Damn, I though my CPU load is to
high, but now I wondering why is yours higher? ;)

-- 
Best regards,
 Konstantin