Hello listmates, We are working on setting up two private networks linked by a public network which is fast (1 Gbit/s) but potentially insecure. Since the hosts on our two networks need to talk to each other, and do so securely, we have decided to use OpenVPN to connect them, making one gateway a server and the other a client. The connectivity part was easy to establish and worked like a charm. The only problem was, and is, performance. We have two old PIII-class machines that are being tested for the role of the gateways. We have put new 1 Gbit NIC's in them and they work find for everything (data transmission, DHCP, DNS, routing) except the VPN. When traffic goes through the VPN the OpenVPN process goes to 99% CPU on the server, about 70% CPU on the client and the effective transmission rate goes down to about 6 MB/s whereas in non-VPN mode it can be as high as 50+ MB/s (the top for the 1 Gbit/s is, obviously, 125 MB/s hence with the VPN we are down to about 5% of the capacity). While this may be usable we would like to hope we can do better. Hence the following questions: 1) Have you used OpenVPN in a similar setup? 2) If so what sort of performance did you see? 3) What kind of equipment did you use? Personally, I'd like to hope that if we find VPN-enabled gateways with more processing power we'd get drastically better performance. So if you have data to confirm or deny that please share it. Thanks. Boris.
Sounds like you need boxes with more cpu. What's the output when you type: cat /proc/cpuinfo into a shell prompt? What kind of traffic goes over the link? Lots of low bandwidth connections? A few high bandwidth connections? Lots of high bandwidth connections? Some combination of the above? Just testing a bunch of boxes with a variety of specs may give you a better answer the pondering the mysteries of the specifications.
On 8/19/2010 10:29 AM, Boris Epstein wrote:> Hello listmates, > > We are working on setting up two private networks linked by a public > network which is fast (1 Gbit/s) but potentially insecure. Since the > hosts on our two networks need to talk to each other, and do so > securely, we have decided to use OpenVPN to connect them, making one > gateway a server and the other a client. The connectivity part was > easy to establish and worked like a charm. The only problem was, and > is, performance. > > We have two old PIII-class machines that are being tested for the role > of the gateways. We have put new 1 Gbit NIC's in them and they work > find for everything (data transmission, DHCP, DNS, routing) except the > VPN. When traffic goes through the VPN the OpenVPN process goes to 99% > CPU on the server, about 70% CPU on the client and the effective > transmission rate goes down to about 6 MB/s whereas in non-VPN mode it > can be as high as 50+ MB/s (the top for the 1 Gbit/s is, obviously, > 125 MB/s hence with the VPN we are down to about 5% of the capacity). > > While this may be usable we would like to hope we can do better. Hence > the following questions: > > 1) Have you used OpenVPN in a similar setup? > > 2) If so what sort of performance did you see? > > 3) What kind of equipment did you use? > > Personally, I'd like to hope that if we find VPN-enabled gateways with > more processing power we'd get drastically better performance. So if > you have data to confirm or deny that please share it.I have an OpenVPN gateway running on an old PII-400 machine with 256M RAM. It works fine for what we need. I have never measured throughput or CPU usage. I would say that if your CPU is going to 99% when you use the VPN, you would definitely benefit from a faster system. I would suspect that any P4 or higher system would work fine, but maybe someone else that actually uses a high-speed VPN connection could give you a more accurate spec. -- Bowie
> We are working on setting up two private networks linked by a public > network which is fast (1 Gbit/s) but potentially insecure. Since the > hosts on our two networks need to talk to each other, and do so > securely, we have decided to use OpenVPN to connect them, making one > gateway a server and the other a client. The connectivity part was > easy to establish and worked like a charm. The only problem was, and > is, performance. > > We have two old PIII-class machines that are being tested for the role > of the gateways. We have put new 1 Gbit NIC's in them and they workWhy you are using such a antique hardware? -- Eero
On Thu, 19 Aug 2010, Boris Epstein wrote:> While this may be usable we would like to hope we can do better. Hence > the following questions: > > 1) Have you used OpenVPN in a similar setup?Similar, though it's only a 100Mb/s link.> 2) If so what sort of performance did you see?Pretty much maximum throughput for a 100Mb circuit. wget, for instance, reports 11.0MB/s.> 3) What kind of equipment did you use?The gateways have Xeon E5405 CPUs and run CentOS 5. -- Paul Heinlein <> heinlein at madboa.com <> http://www.madboa.com/
On 08/19/10 7:29 AM, Boris Epstein wrote:> We have two old PIII-class machines that are being tested for the role > of the gateways. We have put new 1 Gbit NIC's in them and they work > find for everything (data transmission, DHCP, DNS, routing) except the > VPN. When traffic goes through the VPN the OpenVPN process goes to 99% > CPU on the server, about 70% CPU on the client and the effective > transmission rate goes down to about 6 MB/s whereas in non-VPN mode it > can be as high as 50+ MB/s (the top for the 1 Gbit/s is, obviously, > 125 MB/s hence with the VPN we are down to about 5% of the capacity).encryption is CPU intensive. you might get a little better throughput if you can force Blowfish cypher over whatever else as it is somewhat lower in CPU overhead, and still considered decently strong, but you still likely won't get wire speed. heck, you can't get wirespeed doing a simple http bulk transfer over gigE with a p3. note going from P3 to early P4, you need to at least double the clock speed to get about the same performance.. I wouldn't touch a p4 under 2.8Ghz for this. The Core line of CPUs are considerably faster per clock than the P4's. AMD's athlon64/opterons perform much better than P4's at the same clock speeds.
On Thu, Aug 19, 2010, Boris Epstein wrote:>Hello listmates, > >We are working on setting up two private networks linked by a public >network which is fast (1 Gbit/s) but potentially insecure. Since the >hosts on our two networks need to talk to each other, and do so >securely, we have decided to use OpenVPN to connect them, making one >gateway a server and the other a client. The connectivity part was >easy to establish and worked like a charm. The only problem was, and >is, performance. > >We have two old PIII-class machines that are being tested for the role >of the gateways. We have put new 1 Gbit NIC's in them and they work >find for everything (data transmission, DHCP, DNS, routing) except the >VPN. When traffic goes through the VPN the OpenVPN process goes to 99% >CPU on the server, about 70% CPU on the client and the effective >transmission rate goes down to about 6 MB/s whereas in non-VPN mode it >can be as high as 50+ MB/s (the top for the 1 Gbit/s is, obviously, >125 MB/s hence with the VPN we are down to about 5% of the capacity). > >While this may be usable we would like to hope we can do better. Hence >the following questions: > >1) Have you used OpenVPN in a similar setup?We have a client with offices in 4 cities using a Windows application with remote access (which performs horribly compared to their previous *nix applictioan :-). The main site is in Kansas City, the other three in Texas, and the performance is good enough that people aren't complaining -- much as many prefer the old app.>2) If so what sort of performance did you see?The client is happy, particularly since their software vendor wanted them to get $2,500 Cisco routers for each office, and the Linux boxes cost a lot less including setup and configuration. Frankly I was amazed that this was adequate for use with Window remote access over relatively slow links with the T1 in KC probably being the potential bottleneck with 3 offices connecting to it.>3) What kind of equipment did you use?Each office has a T1 connection. The KC Linux machine is a general purpose machine doing e-mail, user storage, etc. NAT gateway for the LAN, as well as the OpenVPN with a single Intel(R) Core(TM)2 Duo CPU E4500 @ 2.20GHz and 2GB RAM. The remote office machines are also NAT gateways for each office's LAN are are running single processor Intel(R) Atom(TM) CPU 330 @ 1.60GHz with 2GB of RAM. These are in small chassis, are very quiet, and seem to work very well. These systems with 80GB SATA drives cost us just under $500 each a couple of years ago, and a bit less today. All are running CentOS 5.x. Bill -- INTERNET: bill at celestial.com Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way Voice: (206) 236-1676 Mercer Island, WA 98040-0820 Fax: (206) 232-9186 Skype: jwccsllc (206) 855-5792 Our Foreign dealings are an Open Book, generally a Check Book. Will Rogers
On 08/19/2010 03:29 PM, Boris Epstein wrote:> 1) Have you used OpenVPN in a similar setup?you nver mentioned what the setup is. post your server.conf as a start if you want to compare notes.> > 2) If so what sort of performance did you see?a very old tyan S3xxx rig, 2x opteron 246HE's, easily sustain about 150 connections and can saturate a 300mbps link ( its a gigabit local port, but only 300mbps makes it out of the rack )> 3) What kind of equipment did you use?hardware rng( via padlock ) and careful selection of openssl cypher + interfaces. - KB
On 08/19/2010 07:29 AM, Boris Epstein wrote:> 1) Have you used OpenVPN in a similar setup?For the hell of it, I ran openvpn on two hosts in a 1Gbps LAN.> 2) If so what sort of performance did you see?I was able to copy a 707788800 byte ISO in 55 seconds using the default BF-CBC cipher. I used "nc" to transmit the file. The same file over the non-vpn route took only 17 seconds. CPU utilization of the 'nc' process was rather high in the second test, which probably indicates that nc was sending data in fairly small buffers. I'm at a loss to explain it, but scp was actually much faster, despite encrypting the data twice over (though that was happening on different CPUs, so probably didn't slow things down much). scp copied the same file in 27 seconds over the VPN, and 20 seconds over the LAN.> 3) What kind of equipment did you use?Two Dell Inspiron 531s systems. One running CentOS 5, the other Fedora 13. Both have AMD Athlon64 X2 4000+ CPUs. Both have D-Link DGE-530T Gig-E network adapters. The MTU between the systems is 9000.
See below... -----Original Message----- From: centos-bounces at centos.org [mailto:centos-bounces at centos.org] On Behalf Of Gordon Messmer Sent: Saturday, August 21, 2010 12:03 AM To: CentOS mailing list Subject: Re: [CentOS] OpenVPN throughput On 08/20/2010 10:19 AM, Bill Campbell wrote:> > Comparisons with gigabit networks seems a pointless given that most > VPNs will be used over the Internet which limits bandwidth, how fast > is fast enough?Boris said in his first email that the link between the two networks was 1 Gps. _______________________________________________ Perhaps a bit late on this thread, but i thought of adding my $0.02 ... Last year i've been doing some experiments with openvpn. Just as the O.P. I was curious about sustainable throughput, and was disapointed about the results To obtain maximum resulst, i did: - use two rather heavy machines (HP DL380-G6, dual quad core) - two dedicated 10Gb-nic's - cross-connect both nics - DISABLE openvpn-debug (as it is VERY cpu expensive) - raise MTU to 4K Bottleneck was (in my case) the openvpn-process, that was running 100% on a single core, While network was not saturated. So for max throughput, it is probably strongswan (ipsec) or hw-encryption [or both] hw ______________________________________________________________________ Dit bericht kan informatie bevatten die niet voor u is bestemd. Indien u niet de geadresseerde bent of dit bericht abusievelijk aan u is toegezonden, wordt u verzocht dat aan de afzender te melden en het bericht te verwijderen. De Staat aanvaardt geen aansprakelijkheid voor schade, van welke aard ook, die verband houdt met risico's verbonden aan het elektronisch verzenden van berichten. This message may contain information that is not intended for you. If you are not the addressee or if this message was sent to you by mistake, you are requested to inform the sender and delete the message. The State accepts no liability for damage of any kind resulting from the risks inherent in the electronic transmission of messages.