We are looking at buying a server to do virtualisation on, hopefully Xen assuming all my testing goes well. Has anyone done any benchmarks on CPU''s from both AMD and Intel for Xen HVM scenarios? AMD''s are a bit cheaper but in the line of servers we are looking at, only the Intel processors come in a quad core configuration, which would mean we could buy a single quad core processor now and upgrade later when required. With the AMD processors, there will be no room to upgrade if we start with 4 processors. All comments appreciated! Thanks James _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> -----Original Message----- > From: xen-users-bounces@lists.xensource.com > [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of > James Harper > Sent: 24 May 2007 02:22 > To: xen-users@lists.xensource.com > Subject: [Xen-users] AMD or Intel? > > We are looking at buying a server to do virtualisation on, > hopefully Xen > assuming all my testing goes well. > > Has anyone done any benchmarks on CPU''s from both AMD and > Intel for Xen > HVM scenarios? AMD''s are a bit cheaper but in the line of > servers we are > looking at, only the Intel processors come in a quad core > configuration, > which would mean we could buy a single quad core processor now and > upgrade later when required. With the AMD processors, there will be no > room to upgrade if we start with 4 processors.To quote from a previous post on the same subject from me: <quote> As you can see from my mail-address, I''m certainly biased... However, wearing my patented "objective-tin-foil-hat" to make me a bit more unbiased, I''ll try to say this: There is VERY LITTLE technical "external" differences between the AMD and Intel virtualization technologies, so from that perspective one isn''t noticably better than the other. </quote> I doubt that there is a huge difference between AMD and Intel on the basis that there''s very similar number of intercepts and the number of clocks for a VMExit+Vmentry are roughly the same [1] - and the majority of intercepts are: - Page-fault - this is often the biggest intercept count by far, because it''s responsible for both MMIO and shadow-paging. - IOIO operations (disk, network, PIC, APIC, timers and other HW devices - depends quite a bit on what the guest does). Both of these type of intercepts are fairly complex to handle, as it involves both interpreting the instructions that the guest was executing, and then performing the adequate operations. For MMIO and IOIO operations, it also often leads to qemu-dm calls. There are some minor differences between AMD and Intel in how and what gets intercepted, but it''s more on the "rare" side, rather than the common ones. Intercepts that happen 1, 2 or 3% of the time isn''t the ones that make the big performance impacts! So what I''m trying to say is that the difference in the implementation of the virtualization in the two processors will be a minor part of the overall performance of the virtual machine overall. There is, however, a major difference in the overall architecture, which DOES affect the performance of the virtual machine: the AMD processors have built-in memory controller, whilst the Intel products have an external memory controller. This helps more on virtual machines than it does on other processor applications, because all sorts of extra memory requirements (hypervisor itself, multiple guests sharing the same processor, shadow page-tables [that is, twice as much memory used for page-tables], emulation of HW devices using a little bit of extra memory - all of these add up a little bit at a time, compared to the bare-metal solution). Admittedly, AMD doesn''t have a quad-core today, but we''re not THAT far away[2] from releasing our quad-core products, so if you need to upgrade later on, you should be able to replace your current dual-core(s) with new quad-core processor(s) in the same socket(s). [1] From our internal measurement a while back, the AMD processor was a tad faster - but that was a while back and it''s not an official benchmark that got published, just some measurements one of my collegues did "on his spare time". [2] I wish I could tell you exactly when we are releasing the new generation processors, but for several reasons I can''t tell you. The major reason here is that I simply don''t know, as I don''t work for the marketing department, but rather work for the engineering side where processors and software components are developed. But if I did know, I wouldn''t be allowed to tell you anyways. -- Mats> > All comments appreciated! > > Thanks > > James > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> > -- > Mats >Mats, Thankyou very much for the response. Much appreciated! James _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Thu, May 24, 2007 at 08:26:26PM +1000, James Harper wrote:> > > > -- > > Mats > > > > Mats, > > Thankyou very much for the response. Much appreciated!yep very informative> > James > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Petersson, Mats wrote:> > > >> -----Original Message----- >> From: xen-users-bounces@lists.xensource.com >> [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of >> James Harper >> Sent: 24 May 2007 02:22 >> To: xen-users@lists.xensource.com >> Subject: [Xen-users] AMD or Intel? >> >> We are looking at buying a server to do virtualisation on, >> hopefully Xen >> assuming all my testing goes well. >> >> Has anyone done any benchmarks on CPU''s from both AMD and >> Intel for Xen >> HVM scenarios? AMD''s are a bit cheaper but in the line of >> servers we are >> looking at, only the Intel processors come in a quad core >> configuration, >> which would mean we could buy a single quad core processor now and >> upgrade later when required. With the AMD processors, there will be no >> room to upgrade if we start with 4 processors. >> > > To quote from a previous post on the same subject from me: <quote> > As you can see from my mail-address, I''m certainly biased... > > However, wearing my patented "objective-tin-foil-hat" to make me a bit > more unbiased, I''ll try to say this: > There is VERY LITTLE technical "external" differences between the AMD > and Intel virtualization technologies, so from that perspective one > isn''t noticably better than the other. > </quote> > > I doubt that there is a huge difference between AMD and Intel on the > basis that there''s very similar number of intercepts and the number of > clocks for a VMExit+Vmentry are roughly the same [1] - and the majority > of intercepts are: > - Page-fault - this is often the biggest intercept count by far, because > it''s responsible for both MMIO and shadow-paging. > - IOIO operations (disk, network, PIC, APIC, timers and other HW devices > - depends quite a bit on what the guest does). > > Both of these type of intercepts are fairly complex to handle, as it > involves both interpreting the instructions that the guest was > executing, and then performing the adequate operations. For MMIO and > IOIO operations, it also often leads to qemu-dm calls. > > There are some minor differences between AMD and Intel in how and what > gets intercepted, but it''s more on the "rare" side, rather than the > common ones. Intercepts that happen 1, 2 or 3% of the time isn''t the > ones that make the big performance impacts! > > So what I''m trying to say is that the difference in the implementation > of the virtualization in the two processors will be a minor part of the > overall performance of the virtual machine overall. > > There is, however, a major difference in the overall architecture, which > DOES affect the performance of the virtual machine: the AMD processors > have built-in memory controller, whilst the Intel products have an > external memory controller. This helps more on virtual machines than it > does on other processor applications, because all sorts of extra memory > requirements (hypervisor itself, multiple guests sharing the same > processor, shadow page-tables [that is, twice as much memory used for > page-tables], emulation of HW devices using a little bit of extra memory > - all of these add up a little bit at a time, compared to the bare-metal > solution). >On a somewhat related topic, my understanding is that Barcelona/K10 will provide hardware (IO-MMU?) that will allow expansion cards (PCI, AGP, PCIe, etc.) to be virtualised. Currently, if an expansion card were to attempt to access RAM in a guest it would be accessing the RAM at that address in the host, i.e., memory accesses in guests are not mapped to the appropriate memory in the host. Is it true that Barcelona/K10 will provide functionality to solve this problem? If so, has any work been done on supporting this hardware in Xen? I would dearly love to reduce my impact on the environment via my energy bill by plugging more video cards into my server in my cellar. :-) Steve. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> On a somewhat related topic, my understanding is that > Barcelona/K10 will > provide hardware (IO-MMU?) that will allow expansion cards (PCI, AGP, > PCIe, etc.) to be virtualised. Currently, if an expansion card were to > attempt to access RAM in a guest it would be accessing the RAM at that > address in the host, i.e., memory accesses in guests are not mapped to > the appropriate memory in the host. Is it true that Barcelona/K10 will > provide functionality to solve this problem? If so, has any work been > done on supporting this hardware in Xen?Sorry to disappoint, the upcoming "barcelona" core will have some improvements to the AMD-V implementation, in particular the nested paging feature, but I can clearly say that it doesn''t have IOMMU. Nested paging allows the guest to run without intercepts of the page-table-updates, avoiding the code handling "shadow paging" altogether. It will speed up most HVM guests, particularly those that do a lot of page-table updates (such as creating/destroying processes a lot - kernel compile for example). The work to support IOMMU in Xen will happen when we''re closer to having hardware with support for this. I''m not entirely sure (nor could I really tell anyone anyways) when the IOMMU will be available, but it''s very unlikely to happen this year, I would think. -- Mats _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hello Mats, Is the IOMMU the one mentioned in this artikle? http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/34434.pdf http://www.amd.com/us-en/Corporate/VirtualPressRoom/0,,51_104_543~117440,00.html If so, in which processor/core will it be available? Regards, Joop Boonen. See also: http://www.xensource.com/files/xs0106_xen_iommu.pdf What does in this artikle http://www.theinquirer.net/default.aspx?article=16879 the IOMMU mean? <quote> Software IOTLB Intel® EM64T does not support an IOMMU in hardware while AMD64 processors do. This means that physical addresses above 4GB (32 bits) cannot reliably be the source or destination of DMA operations. Therefore, the Red Hat Enterprise Linux 3 Update 2 kernel "bounces" all DMA operations to or from physical addresses above 4GB to buffers that the kernel pre-allocated below 4GB at boot time. This is likely to result in lower performance for IO-intensive workloads for Intel® EM64T as compared to AMD64 processors. </quote> On Fri, May 25, 2007 12:51 pm, Petersson, Mats wrote:>> On a somewhat related topic, my understanding is that >> Barcelona/K10 will >> provide hardware (IO-MMU?) that will allow expansion cards (PCI, AGP, >> PCIe, etc.) to be virtualised. Currently, if an expansion card were to >> attempt to access RAM in a guest it would be accessing the RAM at that >> address in the host, i.e., memory accesses in guests are not mapped to >> the appropriate memory in the host. Is it true that Barcelona/K10 will >> provide functionality to solve this problem? If so, has any work been >> done on supporting this hardware in Xen? > > Sorry to disappoint, the upcoming "barcelona" core will have some > improvements to the AMD-V implementation, in particular the nested > paging feature, but I can clearly say that it doesn''t have IOMMU. > > Nested paging allows the guest to run without intercepts of the > page-table-updates, avoiding the code handling "shadow paging" > altogether. It will speed up most HVM guests, particularly those that do > a lot of page-table updates (such as creating/destroying processes a lot > - kernel compile for example). > > The work to support IOMMU in Xen will happen when we''re closer to having > hardware with support for this. > > I''m not entirely sure (nor could I really tell anyone anyways) when the > IOMMU will be available, but it''s very unlikely to happen this year, I > would think. > > -- > Mats > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> -----Original Message----- > From: Joop Boonen [mailto:joop_boonen@web.de] > Sent: 25 May 2007 13:50 > To: Petersson, Mats > Cc: lists; xen-users@lists.xensource.com > Subject: RE: [Xen-users] AMD or Intel? > > Hello Mats, > > Is the IOMMU the one mentioned in this artikle? > http://www.amd.com/us-en/assets/content_type/white_papers_and_ > tech_docs/34434.pdf > http://www.amd.com/us-en/Corporate/VirtualPressRoom/0,,51_104_ > 543~117440,00.htmlYes.> > If so, in which processor/core will it be available?As I said earlier, I don''t know this, and even if I did know, I''d have to shoot you if I told you ;-)> > Regards, > > Joop Boonen. > > > See also: > http://www.xensource.com/files/xs0106_xen_iommu.pdfYes, that''s working on a IOMMU model that is produced by IBM, used in some of their high-end x86 and PowerPC servers as far as I know.> > > What does in this artikle > http://www.theinquirer.net/default.aspx?article=16879 the IOMMU mean?IOMMU in this case isn''t a "real" IOMMU, it''s using the GART (Graphics Aperture Routing Table), which can indeed be used to remap IO-space, but there is at least one problem with it: You can only remap one guest physical address to one machine physical address for all PCI devices. Let''s say you want Xen to use a PCI device for Dom0, and another from DomU, then the IOMMU would have to remap DomU memory accesses, but not Dom0. If the guest physical memory range overlaps the machine range for Dom0, then Dom0''s PCI accesses would potentially be translated and thus use the wrong area of memory. There may be other problems too, but this one will prevent it from doing what the real IOMMU does. The "real" IOMMU is intended to allow different devices or groups of devices to have separate "guest to machine" mappings, and also allow protecting one device from using the others memory and such (so also gives increased security for the system). -- Mats _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users