Hello ! I have a performance issue with Kernel 2.6.X and policy match support as suggested in http://shorewall.net/IPSEC-2.6.html. My IPSEC performance doesn''t exeed about 30kbyte/sec even if my downlink is 1024kbit/sec and should reach more than 100kbyte/sec. No, its not the cpu''s performance (AMD Barton 2500+) and no it''s not the gateway (CELERON 600 Mhz) on the remote side. If I disable ipsec in my /etc/shorewall/hosts which looks like this: #ZONE HOST(S) OPTIONS net ppp0:0.0.0.0/0 gw ppp0:192.168.1.0/24,X.X.X.X/26 ipsec to #ZONE HOST(S) OPTIONS net ppp0:0.0.0.0/0 gw ppp0:192.168.1.0/24,X.X.X.X/26 I get throughput of about 100kbyte/sec. I found out about this problem because I was using SuSE 9.1/9.2 before and had this bad performance, too. (SuSE comes with policy match support in its standard kernels) Now I installed additionally debian sarge with kernel 2.6.8 (no policy match possible) and was impressed about my ipsec performance. After a hard way of upgrading the debian kernel to 2.6.10 and patching in policy match support, setting my shorewall back to use it, I''ve got the same bad performance !!! I think the netfilter ipsec patches suck ! Did anybody do something like me and did notice the same performance degradation ? Shorewall is 2.2.0RC5, should be upgraded today... I use Openswan 2.3.0 on both distros. ESP was set tried with AES and Blowfish, doesn''t matter. -- __________________________________________________ Ralf Schenk fon (02 41) 9 91 21-0 fax (02 41) 9 91 21-59 rs@databay.de Databay AG Hüttenstraße 7 D-52068 Aachen www.databay.de Databay - einfach machen. _________________________________________________
Ralf Schenk wrote:> > Did anybody do something like me and did notice the same performance > degradation ? >No -- I haven''t measured my performance but I''ve been using IPSEC policy match with Shorewall for six months without even suspecting that I had a performance problem. FWIW, if your kernel and iptables contain policy match, I don''t understand how your IPSEC works at all when you remove ''ipsec'' from the /etc/shorewall/hosts file entry; if everyting is set up properly, it shouldn''t work at all. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
Tom Eastep schrieb:> Ralf Schenk wrote: > > >>Did anybody do something like me and did notice the same performance >>degradation ? >> > > > No -- I haven''t measured my performance but I''ve been using IPSEC policy > match with Shorewall for six months without even suspecting that I had a > performance problem. > > FWIW, if your kernel and iptables contain policy match, I don''t > understand how your IPSEC works at all when you remove ''ipsec'' from the > /etc/shorewall/hosts file entry; if everyting is set up properly, it > shouldn''t work at all.That''s true. Only if I boot into my kernel that doesn''t contain policy match support I can use it without. Throughput is 3times higher without policy match ! -- __________________________________________________ Ralf Schenk fon (02 41) 9 91 21-0 fax (02 41) 9 91 21-59 rs@databay.de Databay AG Hüttenstraße 7 D-52068 Aachen www.databay.de Databay - einfach machen. _________________________________________________ Diese E-Mail und etwa angehängte Dateien enthalten vertrauliche Informationen und sind ausschließlich für den Adressaten bestimmt. Sollten Sie irrtümlich diese E-Mail erhalten haben, bitten wir Sie, uns darüber unter info@databay.de zu informieren und die E-Mail ungelesen an uns zurückzusenden und aus Ihrem System zu löschen. This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify info@databay.de. If you are not the named recipient, you should return this message without reading further and delete it from your system.
Ralf Schenk wrote:> > That''s true. Only if I boot into my kernel that doesn''t contain policy > match support I can use it without. Throughput is 3times higher without > policy match ! >You might try disabling policy match (rmmod ipt_policy and rename the ipt_policy.ko file in /lib/modules/`uname -r`/kernel/net/ipv4/netfilter). Now use your Shorewall configuration without ''ipsec'' -- do you see good performance? -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
Tom Eastep wrote:> Ralf Schenk wrote: > > >>Did anybody do something like me and did notice the same performance >>degradation ? >> > > > No -- I haven''t measured my performance but I''ve been using IPSEC policy > match with Shorewall for six months without even suspecting that I had a > performance problem. >Here is what I can measure. Test involves ''cp'' over NFS. First run is with IPSEC, Shorewall, IPSEC-Netfilter. tipper:~ # time cp /download/linux-2.6.8.tar . real 0m30.789s user 0m0.030s sys 0m24.035s This one is with no IPSEC or iptables at all: tipper:~ # time cp /download/linux-2.6.8.tar . real 0m19.493s user 0m0.093s sys 0m10.766s So, the total impact on throughput of IPSEC/Netfilter/Policy Match is roughly 33%. Note: I unmounted/remounted the remote filesystem between runs to avoid client-side caching effects. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
Tom Eastep wrote:> Tom Eastep wrote:> > Here is what I can measure. Test involves ''cp'' over NFS. First run is > with IPSEC, Shorewall, IPSEC-Netfilter. > > tipper:~ # time cp /download/linux-2.6.8.tar . > > real 0m30.789s > user 0m0.030s > sys 0m24.035s > > This one is with no IPSEC or iptables at all: > > tipper:~ # time cp /download/linux-2.6.8.tar . > > real 0m19.493s > user 0m0.093s > sys 0m10.766s > > So, the total impact on throughput of IPSEC/Netfilter/Policy Match is > roughly 33%. > > Note: I unmounted/remounted the remote filesystem between runs to avoid > client-side caching effects. >This was using a cross-over cable between 100bps NICs. The file size was 200MB, so the transfer rates were: 200,000,000 / 31 = 6,451,612 Bps. 200,000,000 / 19.5 = 10,256,510 Bps. So there was roughly a 40% decrease in the transfer rate. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
--On Sunday, January 30, 2005 17:59 +0100 Ralf Schenk <rs@databay.de> wrote:> Hello ! > > I have a performance issue with Kernel 2.6.X and policy match support as > suggested in http://shorewall.net/IPSEC-2.6.html. My IPSEC performance > doesn''t exeed about 30kbyte/sec even if my downlink is 1024kbit/sec and > should reach more than 100kbyte/sec. > > No, its not the cpu''s performance (AMD Barton 2500+) and no it''s not the > gateway (CELERON 600 Mhz) on the remote side.I disagree. Crypto is pretty hard on those little celerons. HEll I see massive performance hits using scp/ssh on Piii600s and 1gig machines. Celerons are not performance CPUs. Add to that the ppp overhead, I can easily see it being the CPU. Have you ever run top or vmstat and watched the CPU idle % during any of this?> Shorewall is 2.2.0RC5, should be upgraded today... > I use Openswan 2.3.0 on both distros. ESP was set tried with AES and > Blowfish, doesn''t matter.Crypto is hard, takes a LOT of CPU. Try running some throughput measurements with vmstat on both ends and record their output. (say vmstat 5 over the course of 30sec-1minute with traffic flowing first in one direction, then the other).
--On Sunday, January 30, 2005 15:11 -0800 Tom Eastep <teastep@shorewall.net> wrote:> This was using a cross-over cable between 100bps NICs. > > The file size was 200MB, so the transfer rates were: > > 200,000,000 / 31 = 6,451,612 Bps. > 200,000,000 / 19.5 = 10,256,510 Bps. > > So there was roughly a 40% decrease in the transfer rate.Which matches very close to a bunch of benchmarks I did a while back with IPSec. You need a really beefy CPU in order to do crypto. OR hardware acceleration.
Michael Loftis wrote:> > Which matches very close to a bunch of benchmarks I did a while back > with IPSec. You need a really beefy CPU in order to do crypto. OR > hardware acceleration. >I had neither -- one end was a Mobile Athlon XP 2200+ laptop and the other end was an Athlon XP 2400+. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
Michael Loftis wrote:> > > --On Sunday, January 30, 2005 15:11 -0800 Tom Eastep > <teastep@shorewall.net> wrote: > >> This was using a cross-over cable between 100bps NICs.I of course meant 100Mbps.>> >> The file size was 200MB, so the transfer rates were: >> >> 200,000,000 / 31 = 6,451,612 Bps. >> 200,000,000 / 19.5 = 10,256,510 Bps. >> >> So there was roughly a 40% decrease in the transfer rate. > > > Which matches very close to a bunch of benchmarks I did a while back > with IPSec. >Good. My main point here is that the cost of the IPSEC-netfilter patches is a part of the 40% decrease that I measured and that my decrease is nowhere near the 66% impact that Ralf is experiencing just as a consequence of the patches alone. So I disagree with Ralf''s "the patches suck" observation since I''m unable to reproduce his results. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
Tom Eastep wrote:> > > Good. My main point here is that the cost of the IPSEC-netfilter patches > is a part of the 40% decrease that I measured and that my decrease is > nowhere near the 66% impact that Ralf is experiencing just as a > consequence of the patches alone. > > So I disagree with Ralf''s "the patches suck" observation since I''m > unable to reproduce his results. >I should point out that in my test, when IPSEC is being used, Shorewall is running on both ends of the IPSEC tunnel (IPSEC-netfilter patches and policy match under SuSE kernel 2.6.8-24.11-default). I just ran a second test which copied the tarball in the other direction. Times were: IPSEC: 43.128s None: 35.432s So in this test, the impact of IPSEC was even less. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
Ralf Schenk wrote:> > That''s true. Only if I boot into my kernel that doesn''t contain policy > match support I can use it without. Throughput is 3times higher without > policy match ! >Have you made any progress with this problem, Ralf? -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key