On 3/16/2015 9:20 AM, John-Mark Gurney wrote:> > Since you have at test framework ready, you could generate some flame > graphs[1] using dtrace to help see where things might be having an > impact... > > These are very easy to generate, and posting them would be useful... > > [1] http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.htmlHi, I went through the steps to generate one. What args should I use for dtrace to generate the information that is helpful / useful ? For my setup, I have server1---------apu-------------server2 server1 has an openvpn tunnel to the apu I route server2's IP address across the VPN tunnel, so if I ping from server1 to server2's IP, it goes via the tunnel on the dtrace -x ustackframes=100 -n 'profile-99 /execname == "openvpn" && arg1/ { @[ustack()] = count(); } tick-30s { exit(0); }' -o 10.stacks which generated http://tancsa.com/10.svg ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike at sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/
John-Mark Gurney
2015-Mar-18 21:14 UTC
35-40% performance drop releng9 vs releng10 openvpn
Mike Tancsa wrote this message on Wed, Mar 18, 2015 at 15:49 -0400:> On 3/16/2015 9:20 AM, John-Mark Gurney wrote: > > > > Since you have at test framework ready, you could generate some flame > > graphs[1] using dtrace to help see where things might be having an > > impact... > > > > These are very easy to generate, and posting them would be useful... > > > > [1] http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html > > > Hi, > I went through the steps to generate one. What args should I use for > dtrace to generate the information that is helpful / useful ? For my > setup, I have > > server1---------apu-------------server2 > > server1 has an openvpn tunnel to the apu > I route server2's IP address across the VPN tunnel, so if I ping from > server1 to server2's IP, it goes via the tunnel > > on the > > dtrace -x ustackframes=100 -n 'profile-99 /execname == "openvpn" && > arg1/ { @[ustack()] = count(); } tick-30s { exit(0); }' -o 10.stacks > > which generated > http://tancsa.com/10.svgSo, I would first identify the machine w/ the cpu limited load.. I assume that is apu... Then I would look at where most of the cpu time is being spent, be it openvpn itself, or in the kernel... Most likely it is the kernel, so getting stacks from the kernel would be more useful than the one you generated... Use the command: # dtrace -x stackframes=100 -n 'profile-997 /arg0/ { @[stack()] = count(); } tick-60s { exit(0); }' -o out.kern_stacks Also, another thing you can do is to compare the two using differential flame graphs: http://www.brendangregg.com/blog/2014-11-09/differential-flame-graphs.html Which will highlight where the performances differ... As I've never used OpenVPN before and their docs don't go into saying what it's using.. Is OpenVPN a kernel or userland VPN? Do they use IPSec in the kernel? or are they just using UDP or TCP for their connections? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."