> Update: I went ahead and compiled glibc without TLS / NPTL.For 64 bit Xen you don''t need to worry about TLS/NPTL.> Also, I realized that I wasn''t allocating all my physical > memory to Domain0 for these tests, so I was giving Xen a > disadvantage. It didn''t change much, but those tests have > been re-done. > > Here''s the odd thing: disabling TLS for 64-bit improved > native performance by around 10%, but had no effect on Xen.How did you disable TLS on 64 bit? It''s odd that it would have any effect on native or xen, let alone such a dramatic one. Are you sure about this test?> Native Domain0 Penalty > 64-bit TLS 50.2 65.0 29.5% > 64-bit no TLS 44.8 65.0 45.1% > 32-bit TLS 59.6 59.5 -0.2% > 32-bit no TLS 59.6 60.2 1.0%It looks like we need to investigate x86_64 networking performance. We may well not be getting the pipelining that we do in 32bit, though I can''t immediately think why, Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Pratt wrote:> > > >>Update: I went ahead and compiled glibc without TLS / NPTL. >> >> > >For 64 bit Xen you don''t need to worry about TLS/NPTL. > > > >>Also, I realized that I wasn''t allocating all my physical >>memory to Domain0 for these tests, so I was giving Xen a >>disadvantage. It didn''t change much, but those tests have >>been re-done. >> >>Here''s the odd thing: disabling TLS for 64-bit improved >>native performance by around 10%, but had no effect on Xen. >> >> > >How did you disable TLS on 64 bit? It''s odd that it would have any >effect on native or xen, let alone such a dramatic one. Are you sure >about this test? > > > >> Native Domain0 Penalty >>64-bit TLS 50.2 65.0 29.5% >>64-bit no TLS 44.8 65.0 45.1% >>32-bit TLS 59.6 59.5 -0.2% >>32-bit no TLS 59.6 60.2 1.0% >> >> > >It looks like we need to investigate x86_64 networking performance. We >may well not be getting the pipelining that we do in 32bit, though I >can''t immediately think why, > >This is a bit surprising as most newer cpus (really any amd_64) has plenty of processing power, and the Ethernet adapter is almost always the bottleneck. This was with netperf, but I would not expect a http get of a single file to be much worse.. I have also seen more overhead in ia32 than amd_64. Xenoprofile showed (for me) about a 13% cpu overhead (more than baremetal linux) in amd_64, 3% in xen and about 10% for the sw bridge. On ia32 (intel xeon) this was about 23%, 13% in xen and 10% for the bridge. I guess I should run these over again and check; maybe something have dramatically changed things. You might want to try this test without starting xend, so the network bridge is not created and eth0 is put on it. Perhaps that is an issue. Come to think of it, my tests were not with veth0/vif0.0 -that may indeed be an issue, going through the front/back-end virtual interfaces, then on the bridge, then on eth0. I''ll kick off some tests tomorrow and see what I get. -Andrew _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
I did `apt-get source glibc`, edited debian/sysdeps/amd64.mk to comment out the couple of lines enabling NPTL, then used dpkg-buildpackage to generate new .debs. Your skepticism was warranted; I double-checked my changes to that file, and I had also disabled a line that set -O3 for the compile. This seems to account for the performance difference on native; recompiling with -O3 intact slows things down to normal. So the conclusion is that NPTL / TLS doesn''t make much difference. However, it''s kind of odd that disabling -O3 would make things faster on native, but have no effect on Xen. Alex On Sunday 09 October 2005 04:18 am, Ian Pratt wrote:> > Update: I went ahead and compiled glibc without TLS / NPTL. > > For 64 bit Xen you don''t need to worry about TLS/NPTL. > > > Also, I realized that I wasn''t allocating all my physical > > memory to Domain0 for these tests, so I was giving Xen a > > disadvantage. It didn''t change much, but those tests have > > been re-done. > > > > Here''s the odd thing: disabling TLS for 64-bit improved > > native performance by around 10%, but had no effect on Xen. > > How did you disable TLS on 64 bit? It''s odd that it would have any > effect on native or xen, let alone such a dramatic one. Are you sure > about this test? > > > Native Domain0 Penalty > > 64-bit TLS 50.2 65.0 29.5% > > 64-bit no TLS 44.8 65.0 45.1% > > 32-bit TLS 59.6 59.5 -0.2% > > 32-bit no TLS 59.6 60.2 1.0% > > It looks like we need to investigate x86_64 networking performance. We > may well not be getting the pipelining that we do in 32bit, though I > can''t immediately think why, > > Ian_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> You might want to try this test without starting xend, so the network > bridge is not created and eth0 is put on it. Perhaps that is an issue. > Come to think of it, my tests were not with veth0/vif0.0 -that may > indeed be an issue, going through the front/back-end virtual interfaces, > then on the bridge, then on eth0. I''ll kick off some tests tomorrow and > see what I get.xend was never started. All I did was boot with the Domain0 kernel. No bridges; I''m talking straight to eth0. That answers Nicholas''s question, too; I haven''t done anything besides Domain0 talking to itself. Alex _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel