Hi all,
I found enabling hyperthreading for Xen (the hypervisor layer) would
degrade network performance. I wonder if the synchronization cost of Xen
(the hypervisor) running on multi cores/multiprocessors environment is
high (around 30 %). The benchmark I did is against Intel Pentium 2.4 MHz
Hyperthreaded CPU and Gigabit network (ethernet and switch). I was using
WebStone 2.5 benchmark against Apache 2.0.50-1.0 (FC1 httpd package). I
was also running 2.4.27 for the domains kernel. The result I have:
1. Native:
a. Hyperthread enabled: 370 Mb/sec
b. Hyperthread disabled: 270-290 Mb/sec
2. Domain 0 has the same performance around: 270 Mb/sec
3. Domain 1:
a, Hyperthreaded enabled, domain1 run on different core: 175-185 Mb/sec
b. Hyperthreaded disabled or even if enabled domain 1 is forced to
be in the same Core with domain 0: 255-265 Mb/sec
Please note, I was only sending request to one domain at a time. It
seems when the two domain ran in the same core we had higher
performance. I wonder if any other people has done almost similar test
and have similiar/opposite behavior. Are these behaviors make sense?
Thanks in advance for any comments.
Kuas.
-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader''s Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel
They''ve seen anywhere from a 30% decrease to 30% increase in performance on SPECint with hyperthreading turned on. Bear in mind you''re now sharing a single cache between two processes. For many workloads the trace cache on the P4 was too small to begin with. -Kip On Sat, 30 Oct 2004, kuas wrote:> Hi all, > > I found enabling hyperthreading for Xen (the hypervisor layer) would > degrade network performance. I wonder if the synchronization cost of Xen > (the hypervisor) running on multi cores/multiprocessors environment is > high (around 30 %). The benchmark I did is against Intel Pentium 2.4 MHz > Hyperthreaded CPU and Gigabit network (ethernet and switch). I was using > WebStone 2.5 benchmark against Apache 2.0.50-1.0 (FC1 httpd package). I > was also running 2.4.27 for the domains kernel. The result I have: > 1. Native: > a. Hyperthread enabled: 370 Mb/sec > b. Hyperthread disabled: 270-290 Mb/sec > 2. Domain 0 has the same performance around: 270 Mb/sec > 3. Domain 1: > a, Hyperthreaded enabled, domain1 run on different core: 175-185 Mb/sec > b. Hyperthreaded disabled or even if enabled domain 1 is forced to > be in the same Core with domain 0: 255-265 Mb/sec > > Please note, I was only sending request to one domain at a time. It > seems when the two domain ran in the same core we had higher > performance. I wonder if any other people has done almost similar test > and have similiar/opposite behavior. Are these behaviors make sense? > > Thanks in advance for any comments. > > Kuas. > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Sybase ASE Linux Express Edition - download now for FREE > LinuxWorld Reader''s Choice Award Winner for best database on Linux. > http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xen-devel >------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> I found enabling hyperthreading for Xen (the hypervisor layer) would > degrade network performance. I wonder if the synchronization cost of Xen > (the hypervisor) running on multi cores/multiprocessors environment is > high (around 30 %).If the domain controlling the network interface is on a different (logical) processor to the domain which is currently using it, there is an overhead because inter-processor interrupts (IPIs) must be sent to notify the other CPU when data is ready. Xen''s pipelined, asynchronous device channels attempt to avoid this problem but some workloads (e.g. small packets at a very high rate) will still unavoidably stress the system. We found in tests that hyperthreading improved performance over simply using one core, when running dom0 on one hyperthread and dom1 on another. An IPI is still sent between the two logical CPUs, which still has overhead, however the performace seemed to be improved because context switches were avoided, relative to time-slicing both domains on one CPU. It''s interesting that performance reduced for you when enabling HT. Perhaps the context-swtich vs IPI tradeoff is different for you, either for hardware reasons (we were using 3GHz Xeons, so had a different core, larger caches, etc.) or because of some software change. My $0.02 Mark ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Sat, 30 Oct 2004, Kip Macy wrote:> They''ve seen anywhere from a 30% decrease to 30% increase in performance > on SPECint with hyperthreading turned on. Bear in mind you''re now > sharing a single cache between two processes. For many workloads the > trace cache on the P4 was too small to begin with.Not just 2 processes. 2 whole operating systems. An operating system has a much higher overhead, and you''ll probably get lots of cache trashing. Note, once xen supports SMP, then it might help if it''s scheduler were HT-aware. Also, why not use normal linux schedulers(don''t know if you currently are)? ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel