Has anyone else seen a significant reduction in performance of Java applications running in a domU? We''re seeing a reduction in performance of around 30% when running a Java (Tomcat, etc webapp) in a domU over running it natively -- this is significantly worse than what we see for other applications. Wondering if anyone else has seen this problem (so we know we''re not going nuts), and whether you were able to tune things to make it perform acceptably. In the absence of any solutions, we''re going to have write Xen off on this project, which will be a big shame (and a nasty black mark in Xen''s copybook in the eyes of The Powers That Be). - Matt _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Christopher G. Stach II
2006-Jun-05 03:08 UTC
Re: [Xen-users] Java non-performant in a domU?
Matthew Palmer wrote:> Has anyone else seen a significant reduction in performance of Java > applications running in a domU? We''re seeing a reduction in performance of > around 30% when running a Java (Tomcat, etc webapp) in a domU over running > it natively -- this is significantly worse than what we see for other > applications. > > Wondering if anyone else has seen this problem (so we know we''re not going > nuts), and whether you were able to tune things to make it perform > acceptably. > > In the absence of any solutions, we''re going to have write Xen off on this > project, which will be a big shame (and a nasty black mark in Xen''s copybook > in the eyes of The Powers That Be). > > - Matt > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-usersI run lots of Sun JVMs (1.4.2 and 1.5.0 running Tomcat 5.0, Eclipse 3.1.2 and 3.2, Resin 3.0.14) in VMs and they perform about as well as expected, being that the machine is being used by other VMs and Xen has a little overhead. They''re certainly not as fast as they normally are, but not 30% slower when nothing else is loading the machine. Is it memory, disk, or CPU that isn''t performing as well? -- Christoper G. Stach II _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Sun, Jun 04, 2006 at 10:08:58PM -0500, Christopher G. Stach II wrote:> Matthew Palmer wrote: > > Has anyone else seen a significant reduction in performance of Java > > applications running in a domU? We''re seeing a reduction in performance of > > around 30% when running a Java (Tomcat, etc webapp) in a domU over running > > it natively -- this is significantly worse than what we see for other > > applications. > > > > Wondering if anyone else has seen this problem (so we know we''re not going > > nuts), and whether you were able to tune things to make it perform > > acceptably. > > I run lots of Sun JVMs (1.4.2 and 1.5.0 running Tomcat 5.0, Eclipse > 3.1.2 and 3.2, Resin 3.0.14) in VMs and they perform about as well as > expected, being that the machine is being used by other VMs and Xen has > a little overhead. They''re certainly not as fast as they normally are, > but not 30% slower when nothing else is loading the machine. > > Is it memory, disk, or CPU that isn''t performing as well?That''s the crazy thing -- we''ve been monitoring everything we can, and we can''t find any anomalous bottlenecks. The application is naturally CPU bound, and we''re giving it the same amount of resources it normally gets -- plenty of RAM (never touches swap), all of the system CPUs, and the machine is doing *nothing* else while we''re benchmarking. It''s pegging on the CPUs, running at a nice healthy 350% of CPU (as per xentop, on a 4-way machine), and there isn''t a lot of disk I/O, but we''re only seeing 100 transactions per second (against a baseline of 140 transactions per second on a fully native machine). We''re running AMD64 systems, and we''ve tried with a 64-bit domU userland with a 32-bit JVM (the app doesn''t run in a 64-bin JVM, because of 3rd party libraries that are running -- so much for platform undependence...), and a 32-bit domU userland over the 64-bit kernel. For the 32-bit userland, we''ve used a Xen-friendly patched glibc, to give us TLS without having to emulate segment stuff. Perhaps that''s it -- we need to run a full 32-bit OS (including the kernel) on these AMD64 boxes, and Xen on top of that. - Matt -- <liw> hut.fi has or used to have two nfs servers not-responding and still-trying... don''t know if their dns server was not-found... 4o4 would be then a good name for the web server... endless hours of fun <aj> "did you get a response from 4o4?" "nah, it just 404ed" _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Sunday 04 June 2006 10:53 pm, Matthew Palmer wrote:> That''s the crazy thing -- we''ve been monitoring everything we can, and we > can''t find any anomalous bottlenecks. The application is naturally CPUjust a wild guess, but on a heavily multithreaded and multiprocessor system like yours, its quite possible that a given JVM could use a non-optimal strategy on a Xen system. I think it''s important to test different JVM''s, and different ''bitness'' of them (64bit vs. 32bit). Since 32bit x86 is _SO_ register-starved, and Xen interferes with the infamous TLS trick, you should try to use a 64bit clean system. since you already stated that your software can''t run on a 64bit JVM, maybe you could get some insight by running a benchmark with similar threading and processor requirements, on all combinations of 32bit, 64bit, Xen and non-Xen. in the worst case, you might find that you must delay the Xen deployment until your software gets more 64bit-friendly. or maybe you''ll find that simply changing JVM get''s you the lost performance. -- Javier _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, Jun 05, 2006 at 12:14:55AM -0500, Javier Guerra wrote:> On Sunday 04 June 2006 10:53 pm, Matthew Palmer wrote: > > That''s the crazy thing -- we''ve been monitoring everything we can, and we > > can''t find any anomalous bottlenecks. The application is naturally CPU > > just a wild guess, but on a heavily multithreaded and multiprocessor system > like yours, its quite possible that a given JVM could use a non-optimal > strategy on a Xen system. I think it''s important to test different JVM''s, > and different ''bitness'' of them (64bit vs. 32bit). Since 32bit x86 is _SO_ > register-starved, and Xen interferes with the infamous TLS trick, you should > try to use a 64bit clean system. since you already stated that your software > can''t run on a 64bit JVM, maybe you could get some insight by running a > benchmark with similar threading and processor requirements, on all > combinations of 32bit, 64bit, Xen and non-Xen.Hmm. You''ve given me some more ideas for benchmarking; I''ll try out some different combos (the different JVM idea is a particularly good one) and report back to the list. Thanks a lot for your help, - Matt _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users