I think that we need to tackle the absurd loss of performance that any Windows HVM incurs with more than one processor. I have measured it and it is amazing. I have an application that opens 15 independent network clients, and with one single processor and the Standard PC Hal, it takes a little over a minute to load the 15 clients and also to register with the SIP provider. If I leave that unchanged, I mean "ceteris paribus" , and only change the HAL to ACPI and add 8 processors, then the same operation takes over 10 minutes. Hello!!! I am using the GPLVL drivers from James Harper. Right now I am about to test the MPS -Non-ACPI HAL, but the driver still create a BSOD. Maybe the problem is only ACPI, and not MPS, who knows. In any case, what can we do so any Windows virtual machine can achieve a "normal" performance, I mean, a linear performance when using more processors and not this declining curve that can kill any host. Federico _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
It seems something wrong. It is hard to achieve linear performance with more processors, but it should be at least something "normal". Another choice might be halsign turbogate drivers. I used it half a year ago, and it is stable and performance is really nice. 2008/12/27 Venefax <venefax@gmail.com>> I think that we need to tackle the absurd loss of performance that any > Windows HVM incurs with more than one processor. I have measured it and it > is amazing. I have an application that opens 15 independent network clients, > and with one single processor and the Standard PC Hal, it takes a little > over a minute to load the 15 clients and also to register with the SIP > provider. If I leave that unchanged, I mean "ceteris paribus" , and only > change the HAL to ACPI and add 8 processors, then the same operation takes > over 10 minutes. Hello!!! I am using the GPLVL drivers from James Harper. > Right now I am about to test the MPS –Non-ACPI HAL, but the driver still > create a BSOD. Maybe the problem is only ACPI, and not MPS, who knows. In > any case, what can we do so any Windows virtual machine can achieve a > "normal" performance, I mean, a linear performance when using more > processors and not this declining curve that can kill any host. > > Federico > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
The problem is not SMP, is ACPI. I installed a non-ACPI Hal, but SMP capable, and the performance went right up with 4 virtual processors. I hope the developers can look into this mess. Federico From: Dirk Utterback [mailto:dirk.utterback@gmail.com] Sent: Sunday, December 28, 2008 9:40 PM To: Venefax Cc: xen-devel@lists.xensource.com Subject: Re: [Xen-devel] Windows SMP It seems something wrong. It is hard to achieve linear performance with more processors, but it should be at least something "normal". Another choice might be halsign turbogate drivers. I used it half a year ago, and it is stable and performance is really nice. 2008/12/27 Venefax <venefax@gmail.com> I think that we need to tackle the absurd loss of performance that any Windows HVM incurs with more than one processor. I have measured it and it is amazing. I have an application that opens 15 independent network clients, and with one single processor and the Standard PC Hal, it takes a little over a minute to load the 15 clients and also to register with the SIP provider. If I leave that unchanged, I mean "ceteris paribus" , and only change the HAL to ACPI and add 8 processors, then the same operation takes over 10 minutes. Hello!!! I am using the GPLVL drivers from James Harper. Right now I am about to test the MPS -Non-ACPI HAL, but the driver still create a BSOD. Maybe the problem is only ACPI, and not MPS, who knows. In any case, what can we do so any Windows virtual machine can achieve a "normal" performance, I mean, a linear performance when using more processors and not this declining curve that can kill any host. Federico _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > The problem is not SMP, is ACPI. I installed a non-ACPI Hal, but SMP > capable, and the performance went right up with 4 virtual processors. > > I hope the developers can look into this mess. >Can you have ACPI enabled but APIC disabled, or is that not a valid configuration? Or the other way around, can you have ACPI disabled but APIC enabled? Maybe the APIC emulation is causing a performance loss? James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
I had to disable both, and PAE. Only APIC=0 would not make any difference. I will some further testing with Citrix Xenserver 5, using the same virtual machine and another copy with their vmpd drivers. I bet that there is no difference in performance. It seems to be a Xen architectural issue. Any ideas? -----Original Message----- From: James Harper [mailto:james.harper@bendigoit.com.au] Sent: Sunday, December 28, 2008 9:53 PM To: Venefax; Dirk Utterback Cc: xen-devel@lists.xensource.com Subject: RE: [Xen-devel] Windows SMP> > The problem is not SMP, is ACPI. I installed a non-ACPI Hal, but SMP > capable, and the performance went right up with 4 virtual processors. > > I hope the developers can look into this mess. >Can you have ACPI enabled but APIC disabled, or is that not a valid configuration? Or the other way around, can you have ACPI disabled but APIC enabled? Maybe the APIC emulation is causing a performance loss? James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
What is your Xen and Windows version? I have been using CentOS5, and Windows XP as the guest. It works fine with 2 vcpu for Windows in my setup(apic=1, acpi=1). Dirk On Sun, Dec 28, 2008 at 6:59 PM, Venefax <venefax@gmail.com> wrote:> I had to disable both, and PAE. Only APIC=0 would not make any difference. > I > will some further testing with Citrix Xenserver 5, using the same virtual > machine and another copy with their vmpd drivers. I bet that there is no > difference in performance. It seems to be a Xen architectural issue. Any > ideas? > > -----Original Message----- > From: James Harper [mailto:james.harper@bendigoit.com.au] > Sent: Sunday, December 28, 2008 9:53 PM > To: Venefax; Dirk Utterback > Cc: xen-devel@lists.xensource.com > Subject: RE: [Xen-devel] Windows SMP > > > > > The problem is not SMP, is ACPI. I installed a non-ACPI Hal, but SMP > > capable, and the performance went right up with 4 virtual processors. > > > > I hope the developers can look into this mess. > > > > Can you have ACPI enabled but APIC disabled, or is that not a valid > configuration? > > Or the other way around, can you have ACPI disabled but APIC enabled? > > Maybe the APIC emulation is causing a performance loss? > > James > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 29/12/2008 02:59, "Venefax" <venefax@gmail.com> wrote:> I had to disable both, and PAE. Only APIC=0 would not make any difference. I > will some further testing with Citrix Xenserver 5, using the same virtual > machine and another copy with their vmpd drivers. I bet that there is no > difference in performance. It seems to be a Xen architectural issue. Any > ideas?The problem is almost certainly APIC related. APIC=0 actually has no effect for a multi-processor HVM guest, since APICs are architecturally absolutely required in x86 multi-processor systems. The problem is most likely lots of emulated APIC TPR writes slowing things down. Possible fixes: 1. Run a Windows guest with the ''lazy TPR'' optimisation -- w2k3sp2+, w2k8, vista. Or run 64-bit Windows which will write TPR in a different way which most Intel/AMD CPUs can virtualise efficiently. 2. Run a new enough Intel processor which has automatic TPR handling even for 32-bit Windows guests. 3. Run Citrix drivers which patch Windows to avoid TPR writes. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > I had to disable both, and PAE. Only APIC=0 would not make any > difference. I > > will some further testing with Citrix Xenserver 5, using the same > virtual > > machine and another copy with their vmpd drivers. I bet that thereis no> > difference in performance. It seems to be a Xen architectural issue.Any> > ideas? > > The problem is almost certainly APIC related. APIC=0 actually has no > effect > for a multi-processor HVM guest, since APICs are architecturally > absolutely > required in x86 multi-processor systems. > > The problem is most likely lots of emulated APIC TPR writes slowingthings> down. Possible fixes: > 1. Run a Windows guest with the ''lazy TPR'' optimisation -- w2k3sp2+, > w2k8, > vista. Or run 64-bit Windows which will write TPR in a different waywhich> most Intel/AMD CPUs can virtualise efficiently. > 2. Run a new enough Intel processor which has automatic TPR handlingeven> for 32-bit Windows guests. > 3. Run Citrix drivers which patch Windows to avoid TPR writes. >Can you elaborate on that last point? Does that pass WHQL? I think that ''venefax'' is running Windows 2003sp2... does that exclude ''lazy TPR'' fix as an option or does it need enabling somehow? Thanks James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 29/12/2008 08:20, "James Harper" <james.harper@bendigoit.com.au> wrote:>> 3. Run Citrix drivers which patch Windows to avoid TPR writes.> Can you elaborate on that last point? Does that pass WHQL?The WHQL tests are oblivious to it. It''s just a patching of mmio writes to the APIC TPR register.> I think that ''venefax'' is running Windows 2003sp2... does that exclude > ''lazy TPR'' fix as an option or does it need enabling somehow?If he''s running sp2 then I think TPR problems can be discounted. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> On 29/12/2008 08:20, "James Harper" <james.harper@bendigoit.com.au>wrote:> > >> 3. Run Citrix drivers which patch Windows to avoid TPR writes. > > > Can you elaborate on that last point? Does that pass WHQL? > > The WHQL tests are oblivious to it. It''s just a patching of mmiowrites to> the APIC TPR register. >Looking at the way KVM does it, it appears to detect writes to the TPR register when they are trapped, and then give the DomU (or whatever KVM calls it) the address of the instruction so that the DomU driver can then patch it. Is that what Citrix is doing? Does the current xensource tree have such a mechanism in it? Thanks James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 29/12/2008 08:47, "James Harper" <james.harper@bendigoit.com.au> wrote:>> The WHQL tests are oblivious to it. It''s just a patching of mmio >> writes to the APIC TPR register. > > Looking at the way KVM does it, it appears to detect writes to the TPR > register when they are trapped, and then give the DomU (or whatever KVM > calls it) the address of the instruction so that the DomU driver can > then patch it. Is that what Citrix is doing? Does the current xensource > tree have such a mechanism in it?The result is the same, but there''s no hypervisor component, so none of it is open sourced. You could get similar results by putting static Windows-kernel-specific fixup tables in your drivers, or I''d be open to having a KVM type of interaction between Xen and your GPLPV drivers. Putting the payload in the generic virtual BIOS seemed kind of gross to me. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> On 29/12/2008 08:47, "James Harper" <james.harper@bendigoit.com.au>wrote:> > >> The WHQL tests are oblivious to it. It''s just a patching of mmio > >> writes to the APIC TPR register. > > > > Looking at the way KVM does it, it appears to detect writes to theTPR> > register when they are trapped, and then give the DomU (or whateverKVM> > calls it) the address of the instruction so that the DomU driver can > > then patch it. Is that what Citrix is doing? Does the currentxensource> > tree have such a mechanism in it? > > The result is the same, but there''s no hypervisor component, so noneof it> is open sourced.:)> You could get similar results by putting staticWindows-kernel-specific> fixup tables in your drivers,As in ''write bytes to offset x into kernel function y'', with x depending on the exact kernel build? Wouldn''t the rootkit detectors complain about that?> or I''d be open to having a KVM type of > interaction between Xen and your GPLPV drivers. Putting the payload inthe> generic virtual BIOS seemed kind of gross to me.Is it possible for a virtualised DomU to trap the MMIO write itself, or can it only be trapped by the hypervisor? Btw, is it the vmexit that is slow about these TPR writes, or is it the writes themselves? Thanks James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 29/12/2008 09:03, "James Harper" <james.harper@bendigoit.com.au> wrote:>> You could get similar results by putting static > Windows-kernel-specific >> fixup tables in your drivers, > > As in ''write bytes to offset x into kernel function y'', with x depending > on the exact kernel build? Wouldn''t the rootkit detectors complain about > that?I''m not actually sure on rootkit-detector compatibility with this approach. I know that the built-in tripwire capabilities of some 64-bit Windows versions would be a problem, but then they tend to have lazy TPR, and also they write to the TPR with MOV CR8, which can usually be handled efficiently by HVM-capable CPUs anyway, so no patching is required. Any software-based approach is going to involve patching, so I''d cross the rootkit-detector bridge when/if we come to it. I haven''t heard or read of any complaints in this regard so far.>> or I''d be open to having a KVM type of >> interaction between Xen and your GPLPV drivers. Putting the payload in > the >> generic virtual BIOS seemed kind of gross to me. > > Is it possible for a virtualised DomU to trap the MMIO write itself, or > can it only be trapped by the hypervisor?It could, by trapping all accesses to the APIC page (remove mapping, hook page-fault handler). It''s going to be much easier to get help from the hypervisor!> Btw, is it the vmexit that is slow about these TPR writes, or is it the > writes themselves?Both. As well as the VMEXIT you have a run through the instruction emulator and into the apic device model. It sucks pretty bad if you do it often. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > Is it possible for a virtualised DomU to trap the MMIO write itself,or> > can it only be trapped by the hypervisor? > > It could, by trapping all accesses to the APIC page (remove mapping,hook> page-fault handler). It''s going to be much easier to get help from the > hypervisor!Hmmm... maybe easier with the hypervisor but by trapping writes I could do it all in my drivers. Assuming the hypervisor route, if I understand correctly it could go like this: . develop a mechanism for qemu''s TPR write emulation to also signal my drivers somehow. This signalling mechanism would have to be low overhead, but could also be pretty lazy (if these TPR writes are happening often then it doesn''t matter if we miss a few as we''ll patch them all pretty quickly) . the PV drivers get the signal, perform some validation on the instruction that caused the TPR write to make sure it''s patchable, and then overwrite the instruction with a call to my own routine, making sure that this is done in an SMP safe way and taking care of caching etc... . My own routine does ??? (I haven''t figured this out exactly, but there is plenty of code out there that does it already) Thinking about this some more... could it work the other way around instead: . My drivers publish to qemu (somehow... io space write maybe???) the required patch for the mmio write instruction, eg a patch request might look like: <01> (magic number indicating a TPR write patch) <05> (length) <01><02><03><04><05> (code to replace the TPR write instruction with) . Qemu stores the above in a table, and every time an mmio write instruction for the TPR register occurs which is exactly 5 bytes in length, patch the instruction with the code given> > Btw, is it the vmexit that is slow about these TPR writes, or is itthe> > writes themselves? > > Both. As well as the VMEXIT you have a run through the instruction > emulator > and into the apic device model. It sucks pretty bad if you do itoften. I see. It''s the sum of a whole load of tiny overheads that adds up... Thanks James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 29/12/2008 09:40, "James Harper" <james.harper@bendigoit.com.au> wrote:> Assuming the hypervisor route, if I understand correctly it could go > like this:Either of your described approaches could work. This doesn''t involve qemu at all, since APIC emulation is done within Xen. So it''ll be a new hvm_op hypercall either way. Getting Xen to do the patching isn''t a bad idea. Might also be worth looking at how KVM does it to get some further ideas. I don''t think your patched routine has to be any more complex than filtering out TPR writes which don''t change the TPR value. Again you could check KVM for this. And of course the TPR write in your patch routine should be skipped by the trap-and-patch mechanism. :-)>> Both. As well as the VMEXIT you have a run through the instruction >> emulator >> and into the apic device model. It sucks pretty bad if you do it > often. > > I see. It''s the sum of a whole load of tiny overheads that adds up...Two fairly big overheads. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> On 29/12/2008 09:40, "James Harper" <james.harper@bendigoit.com.au>wrote:> > > Assuming the hypervisor route, if I understand correctly it could go > > like this: > > Either of your described approaches could work. This doesn''t involveqemu> at > all, since APIC emulation is done within Xen. So it''ll be a new hvm_op > hypercall either way. Getting Xen to do the patching isn''t a bad idea. > Might > also be worth looking at how KVM does it to get some further ideas. >Oh. I had a quick look at hw/apic.c and assumed that''s where the action was... guess I''ll look in xen then! Thanks James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Mon, Dec 29, 2008 at 8:14 AM, Keir Fraser <keir.fraser@eu.citrix.com> wrote:> On 29/12/2008 02:59, "Venefax" <venefax@gmail.com> wrote: > >> I had to disable both, and PAE. Only APIC=0 would not make any difference. I >> will some further testing with Citrix Xenserver 5, using the same virtual >> machine and another copy with their vmpd drivers. I bet that there is no >> difference in performance. It seems to be a Xen architectural issue. Any >> ideas? > > The problem is almost certainly APIC related. APIC=0 actually has no effect > for a multi-processor HVM guest, since APICs are architecturally absolutely > required in x86 multi-processor systems. > > The problem is most likely lots of emulated APIC TPR writes slowing things > down. Possible fixes: > 1. Run a Windows guest with the ''lazy TPR'' optimisation -- w2k3sp2+, w2k8, > vista. Or run 64-bit Windows which will write TPR in a different way which > most Intel/AMD CPUs can virtualise efficiently. > 2. Run a new enough Intel processor which has automatic TPR handling even > for 32-bit Windows guests.Does "PTR handling" have a cpu feature flag I can grep for in /proc/cpuinfo, or perhaps there is a list of cpus that support that feature? I run Xen on two systems, a dual Xeon 5420 and a Core 2 Quad, neither seem to suffer from bad performance with SMP Windows hvm''s but most of the windows hvm''s I run are not heavily loaded so I would like to know if my system is likely to suffer from this problem as it would probably become noticible once load increases. thx Andy> 3. Run Citrix drivers which patch Windows to avoid TPR writes. > > -- Keir > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 29/12/2008 21:40, "Andrew Lyon" <andrew.lyon@gmail.com> wrote:> Does "PTR handling" have a cpu feature flag I can grep for in > /proc/cpuinfo, or perhaps there is a list of cpus that support that > feature?No it''s not a CPUID feature and hence not visible via /proc/cpuinfo. Xen could print out if it detects the feature, but it doesn''t currently. :-) -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Mon, Dec 29, 2008 at 9:57 PM, Keir Fraser <keir.fraser@eu.citrix.com> wrote:> On 29/12/2008 21:40, "Andrew Lyon" <andrew.lyon@gmail.com> wrote: > >> Does "PTR handling" have a cpu feature flag I can grep for in >> /proc/cpuinfo, or perhaps there is a list of cpus that support that >> feature? > > No it''s not a CPUID feature and hence not visible via /proc/cpuinfo. Xen > could print out if it detects the feature, but it doesn''t currently. :-)Hmm, that would be very useful. As its not a cpuid feature I guess I will have to check the Intel manuals for each cpu model, do you know what the feature is described as in Intel documentation?> > -- Keir > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 30/12/2008 10:34, "Andrew Lyon" <andrew.lyon@gmail.com> wrote:>> No it''s not a CPUID feature and hence not visible via /proc/cpuinfo. Xen >> could print out if it detects the feature, but it doesn''t currently. :-) > > Hmm, that would be very useful. > > As its not a cpuid feature I guess I will have to check the Intel > manuals for each cpu model, do you know what the feature is described > as in Intel documentation?In the architecture manual it is known as apic-access virtualisation. I don''t know whether it would be a headline feature in a processor''s data sheet. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, Dec 30, 2008 at 11:04 AM, Keir Fraser <keir.fraser@eu.citrix.com> wrote:> On 30/12/2008 10:34, "Andrew Lyon" <andrew.lyon@gmail.com> wrote: > >>> No it''s not a CPUID feature and hence not visible via /proc/cpuinfo. Xen >>> could print out if it detects the feature, but it doesn''t currently. :-) >> >> Hmm, that would be very useful. >> >> As its not a cpuid feature I guess I will have to check the Intel >> manuals for each cpu model, do you know what the feature is described >> as in Intel documentation? > > In the architecture manual it is known as apic-access virtualisation. I > don''t know whether it would be a headline feature in a processor''s data > sheet.Can you point me to the code where apic-access virtualisation is used? Displaying a message might be a nice small improvement for me to figure out and contribute.> > -- Keir > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > As its not a cpuid feature I guess I will have to check the Intel > > manuals for each cpu model, do you know what the feature isdescribed> > as in Intel documentation? > > In the architecture manual it is known as apic-access virtualisation.I> don''t know whether it would be a headline feature in a processor''sdata> sheet.I believe Intel''s marketing moniker for the feature is "FlexPriority". Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 30/12/2008 12:41, "Andrew Lyon" <andrew.lyon@gmail.com> wrote:>> In the architecture manual it is known as apic-access virtualisation. I >> don''t know whether it would be a headline feature in a processor''s data >> sheet. > > Can you point me to the code where apic-access virtualisation is used? > Displaying a message might be a nice small improvement for me to > figure out and contribute.You can make use of the macro cpu_has_vmx_virtualize_apic_accesses to control a printk() at the end of xen/arch/x86/hvm/vmx/vmcs.c:vmx_init_vmcs_config(). -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
James, Travis Betak released a Windows driver which patches Windows TPR accesses. You can find it from http://sourceforge.net/projects/amdvopt/. The code would be useful as a reference. src link: svn co https://amdvopt.svn.sourceforge.net/svnroot/amdvopt amdvopt Thanks, -Wei James Harper wrote:>> On 29/12/2008 08:20, "James Harper" <james.harper@bendigoit.com.au> > wrote: >>>> 3. Run Citrix drivers which patch Windows to avoid TPR writes. >>> Can you elaborate on that last point? Does that pass WHQL? >> The WHQL tests are oblivious to it. It''s just a patching of mmio > writes to >> the APIC TPR register. >> > > Looking at the way KVM does it, it appears to detect writes to the TPR > register when they are trapped, and then give the DomU (or whatever KVM > calls it) the address of the instruction so that the DomU driver can > then patch it. Is that what Citrix is doing? Does the current xensource > tree have such a mechanism in it? > > Thanks > > James > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
That looks like it could be a nice drop-in piece of code! -- Keir On 30/12/2008 16:58, "Wei Huang" <wei.huang2@amd.com> wrote:> James, > > Travis Betak released a Windows driver which patches Windows TPR > accesses. You can find it from http://sourceforge.net/projects/amdvopt/. > The code would be useful as a reference. src link: > > svn co https://amdvopt.svn.sourceforge.net/svnroot/amdvopt amdvopt > > Thanks, > > -Wei > > > James Harper wrote: >>> On 29/12/2008 08:20, "James Harper" <james.harper@bendigoit.com.au> >> wrote: >>>>> 3. Run Citrix drivers which patch Windows to avoid TPR writes. >>>> Can you elaborate on that last point? Does that pass WHQL? >>> The WHQL tests are oblivious to it. It''s just a patching of mmio >> writes to >>> the APIC TPR register. >>> >> >> Looking at the way KVM does it, it appears to detect writes to the TPR >> register when they are trapped, and then give the DomU (or whatever KVM >> calls it) the address of the instruction so that the DomU driver can >> then patch it. Is that what Citrix is doing? Does the current xensource >> tree have such a mechanism in it? >> >> Thanks >> >> James >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>From: James Harper >Sent: Monday, December 29, 2008 4:48 PM > >> On 29/12/2008 08:20, "James Harper" <james.harper@bendigoit.com.au> >wrote: >> >> >> 3. Run Citrix drivers which patch Windows to avoid TPR writes. >> >> > Can you elaborate on that last point? Does that pass WHQL? >> >> The WHQL tests are oblivious to it. It''s just a patching of mmio >writes to >> the APIC TPR register. >> > >Looking at the way KVM does it, it appears to detect writes to the TPR >register when they are trapped, and then give the DomU (or whatever KVM >calls it) the address of the instruction so that the DomU driver can >then patch it. Is that what Citrix is doing? Does the current xensource >tree have such a mechanism in it? >IIRC, based on my understanding months ago when KVM PTR patching was introduced, it only works for UP windows guest. One tricky issue for dynamic instruction patching is how to encode vcpu ID in register, since vcpu ID is the index into per-vcpu shadow vTPR value. Use memory accesses to retrieve vcpu ID would lower down the performance gain. KVM borrows only one bit from vTR (TI?) as vcpu id. Thanks, Kevin _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > James, > > > > Travis Betak released a Windows driver which patches Windows TPR > > accesses. You can find it fromhttp://sourceforge.net/projects/amdvopt/.> > The code would be useful as a reference. src link: > > > > svn co https://amdvopt.svn.sourceforge.net/svnroot/amdvopt amdvopt> > > That looks like it could be a nice drop-in piece of code! >He is taking a table-based approach, which would require ongoing maintenance. I was hoping to develop a more dynamic version, but given that someone else has already put the patch tables together, doing it dynamically suddenly seems like a lot of work :) Is there a similar approach that would work on an Intel system? Are there any other operations like TPR write''s that are done frequently that could benefit from patching? James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>From: James Harper >Sent: Wednesday, December 31, 2008 4:22 PM >> > James, >> > >> > Travis Betak released a Windows driver which patches Windows TPR >> > accesses. You can find it from >http://sourceforge.net/projects/amdvopt/. >> > The code would be useful as a reference. src link: >> > >> > svn co https://amdvopt.svn.sourceforge.net/svnroot/amdvopt >amdvopt> >> >> That looks like it could be a nice drop-in piece of code! >> > >He is taking a table-based approach, which would require ongoing >maintenance. I was hoping to develop a more dynamic version, but given >that someone else has already put the patch tables together, doing it >dynamically suddenly seems like a lot of work :) > >Is there a similar approach that would work on an Intel system?On Intel CPU with FlexPriority support, you don''t need patching guest since TPR accesses would be recognized by hardware for acceleration automatically. But on CPUs without h/w acceleration support, you may expect borrow that overall idea, but instead of patching with LOCK MOV CR0, you would replace it with a piece of code lines to emulate similar acceleration as what h/w is assumed to do. Thanks, Kevin _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> >Is there a similar approach that would work on an Intel system? > > On Intel CPU with FlexPriority support, you don''t need patching > guest since TPR accesses would be recognized by hardware > for acceleration automatically. > > But on CPUs without h/w acceleration support, you may expect > borrow that overall idea, but instead of patching with LOCK MOV > CR0, you would replace it with a piece of code lines to emulate > similar acceleration as what h/w is assumed to do. >Do you have an example :) One thing Keir suggested would be to install the patch to jump to some code which compared the value being written to the TPR register with the value last written, and only perform the actual write if the values are different. I can do that without too much fuss but if there is something faster then even better. Thanks James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>From: James Harper [mailto:james.harper@bendigoit.com.au] >Sent: Wednesday, December 31, 2008 4:35 PM > >> >Is there a similar approach that would work on an Intel system? >> >> On Intel CPU with FlexPriority support, you don''t need patching >> guest since TPR accesses would be recognized by hardware >> for acceleration automatically. >> >> But on CPUs without h/w acceleration support, you may expect >> borrow that overall idea, but instead of patching with LOCK MOV >> CR0, you would replace it with a piece of code lines to emulate >> similar acceleration as what h/w is assumed to do. >> > >Do you have an example :) > >One thing Keir suggested would be to install the patch to jump to some >code which compared the value being written to the TPR >register with the >value last written, and only perform the actual write if the values areThat''s basically what I meant, and also what KVM does today. VM-exit in such case is only proactively requested by vmcall in inserted lines if Xen emulation logic has to be involved.>different. I can do that without too much fuss but if there is >something >faster then even better. >If you compare to VM-exit overhead for every TPR access, above is already far far faster. Of course fewer memory accesses used in inserted lines, less overhead you''ll see then.:-) Thanks, Kevin _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 31/12/2008 08:50, "Tian, Kevin" <kevin.tian@intel.com> wrote:>> different. I can do that without too much fuss but if there is >> something >> faster then even better. >> > > If you compare to VM-exit overhead for every TPR access, above > is already far far faster. Of course fewer memory accesses used > in inserted lines, less overhead you''ll see then.:-)It''ll be plenty good enough, as you''ll be saving probably on the order of 5000 cycles by not exiting and emulating. You''ll be hard pushed to be worse than 5% of that. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, Dec 30, 2008 at 2:41 PM, Keir Fraser <keir.fraser@eu.citrix.com> wrote:> On 30/12/2008 12:41, "Andrew Lyon" <andrew.lyon@gmail.com> wrote: > >>> In the architecture manual it is known as apic-access virtualisation. I >>> don''t know whether it would be a headline feature in a processor''s data >>> sheet. >> >> Can you point me to the code where apic-access virtualisation is used? >> Displaying a message might be a nice small improvement for me to >> figure out and contribute. > > You can make use of the macro cpu_has_vmx_virtualize_apic_accesses to > control a printk() at the end of > xen/arch/x86/hvm/vmx/vmcs.c:vmx_init_vmcs_config(). >Thanks Keir, I added a couple of printk''s as you suggested and I can now see if the feature is supported: Intel(R) Xeon(R) CPU E5420 @ 2.50GHz = (XEN) APIC Access virtualized Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz = (XEN) APIC Access emulated I didnt expect my Xeon system to support it, I''ve read some pdf''s about Intel virtualizaton features and they seemed to suggest it was a new feature on 7xxx Xeons. The results fit the performance I''ve seen, a windows xp 32 bit hvm with 2 cpu''s runs a lot faster on the Xeon 2.5 than on the Core 2.4. Andy> -- Keir > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 31/12/2008 10:08, "Andrew Lyon" <andrew.lyon@gmail.com> wrote:> > Thanks Keir, I added a couple of printk''s as you suggested and I can > now see if the feature is supported: > > Intel(R) Xeon(R) CPU E5420 @ 2.50GHz = (XEN) APIC Access > virtualized > Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz = (XEN) APIC Access emulated > > I didnt expect my Xeon system to support it, I''ve read some pdf''s > about Intel virtualizaton features and they seemed to suggest it was a > new feature on 7xxx Xeons. > > The results fit the performance I''ve seen, a windows xp 32 bit hvm > with 2 cpu''s runs a lot faster on the Xeon 2.5 than on the Core 2.4.I think it is probably worth printing out. I''ll add a patch to xen-unstable. Thanks, Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Dear Gentlemen I have a machine with Intel 7350, 4 quad core. I can test since I have two virtual machines with 8 vcpus, showing a gross overhead that makes them unsuited for business. I use SLES 10 SP2, and I don''t know how to apply the patch, but if somebody can log in and apply it, we can see the results immediately. The issue is affecting me directly. My two VM''s have a VOIP application, very intensive in network and CPU usage. Federico -----Original Message----- From: Keir Fraser [mailto:keir.fraser@eu.citrix.com] Sent: Wednesday, December 31, 2008 5:10 AM To: Andrew Lyon Cc: Venefax; James Harper; Dirk Utterback; xen-devel@lists.xensource.com Subject: Re: [Xen-devel] Windows SMP On 31/12/2008 10:08, "Andrew Lyon" <andrew.lyon@gmail.com> wrote:> > Thanks Keir, I added a couple of printk''s as you suggested and I can > now see if the feature is supported: > > Intel(R) Xeon(R) CPU E5420 @ 2.50GHz = (XEN) APIC Access > virtualized > Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz = (XEN) APIC Accessemulated> > I didnt expect my Xeon system to support it, I''ve read some pdf''s > about Intel virtualizaton features and they seemed to suggest it was a > new feature on 7xxx Xeons. > > The results fit the performance I''ve seen, a windows xp 32 bit hvm > with 2 cpu''s runs a lot faster on the Xeon 2.5 than on the Core 2.4.I think it is probably worth printing out. I''ll add a patch to xen-unstable. Thanks, Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Venefax wrote:> Dear Gentlemen> I have a machine with Intel 7350, 4 quad core. I can test since I have two > virtual machines with 8 vcpus, showing a gross overhead that makes them > unsuited for business. I use SLES 10 SP2, and I don''t know how to apply the > patch, but if somebody can log in and apply it, we can see the results > immediately. The issue is affecting me directly. My two VM''s have a VOIP > application, very intensive in network and CPU usage.If your problem isn''t the previously discussed TPR issues ... What Windows version are you running in your guests? Windows 2000 SMP has serious problems due to the CPU wasting idle loop. We avoid this with a special idler daemon running in the guest. Are you over-committing VCPUS at all (more than one active VCPU per CPU)? We have noticed significant performance degradation with SMP windows once you over-commit VCPUS on the host. This seems to be due to excessive guest spinlock overhead caused by the spinning VCPUS wasting their entire quantum looping for the guest lock at the same time they are preventing the guest lock holding VCPU from running. We have been experimenting with new CPU features for exiting on PAUSE instructions. This can possibly be used to detect the CPU wasting spinners. Have you tried using the Novell shim (Vista/2008 guests)? I''m not sure the shim provides locking enhancements, but we have seen benefits for certain workloads. Are you running PV on HVM drivers? One last area of concern may be distributing callback interrupts to all VCPUS. We still run Xen (SLES10 SP2 based) with a patch to force all guest callback interrupts to VCPU 0. This has consistently improved our SMP guest I/O performance while running PV-on-HVM drivers. Steve> Federico > -----Original Message----- > From: Keir Fraser [mailto:keir.fraser@eu.citrix.com] > Sent: Wednesday, December 31, 2008 5:10 AM > To: Andrew Lyon > Cc: Venefax; James Harper; Dirk Utterback; xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] Windows SMP > > On 31/12/2008 10:08, "Andrew Lyon" <andrew.lyon@gmail.com> wrote: > >> Thanks Keir, I added a couple of printk''s as you suggested and I can >> now see if the feature is supported: >> >> Intel(R) Xeon(R) CPU E5420 @ 2.50GHz = (XEN) APIC Access >> virtualized >> Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz = (XEN) APIC Access > emulated >> I didnt expect my Xeon system to support it, I''ve read some pdf''s >> about Intel virtualizaton features and they seemed to suggest it was a >> new feature on 7xxx Xeons. >> >> The results fit the performance I''ve seen, a windows xp 32 bit hvm >> with 2 cpu''s runs a lot faster on the Xeon 2.5 than on the Core 2.4. > > I think it is probably worth printing out. I''ll add a patch to xen-unstable. > > Thanks, > Keir > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
I use only Windows 2003 R2 SP2 fully patched. Therefore that patch would not apply since it is for Vista (Novell shim (Vista/2008 guests). But I will ask Novell. I would love to try that second patch that would "force all guest callback interrupts to VCPU 0". My setup is to use one core for xen, only, and the rest 15 cores for virtual machines, as instructed by Novell support. I am not over-committing CPU''s. Even if I run one single virtual machine with 8 CPU''s, the performance is awful. It takes 10 mins to load an application that loads in 1.5-2.0 mins with one single CPU. I switched all my 16 virtual machines to Standard PC Hal and that is the only way that the application works. However, it is an administrative nightmare. Instead of managing 16 independent windows machines via VNC, I should be able to manage two, maybe one. Can you send me information regarding both patches mentioned? Federico -----Original Message----- From: Steve Ofsthun [mailto:sofsthun@virtualiron.com] Sent: Monday, January 05, 2009 1:59 PM To: Venefax Cc: ''Keir Fraser''; ''Andrew Lyon''; ''James Harper''; ''Dirk Utterback''; xen-devel@lists.xensource.com Subject: Re: [Xen-devel] Windows SMP Venefax wrote:> Dear Gentlemen> I have a machine with Intel 7350, 4 quad core. I can test since I have two > virtual machines with 8 vcpus, showing a gross overhead that makes them > unsuited for business. I use SLES 10 SP2, and I don''t know how to apply the > patch, but if somebody can log in and apply it, we can see the results > immediately. The issue is affecting me directly. My two VM''s have a VOIP > application, very intensive in network and CPU usage.If your problem isn''t the previously discussed TPR issues ... What Windows version are you running in your guests? Windows 2000 SMP has serious problems due to the CPU wasting idle loop. We avoid this with a special idler daemon running in the guest. Are you over-committing VCPUS at all (more than one active VCPU per CPU)? We have noticed significant performance degradation with SMP windows once you over-commit VCPUS on the host. This seems to be due to excessive guest spinlock overhead caused by the spinning VCPUS wasting their entire quantum looping for the guest lock at the same time they are preventing the guest lock holding VCPU from running. We have been experimenting with new CPU features for exiting on PAUSE instructions. This can possibly be used to detect the CPU wasting spinners. Have you tried using the Novell shim (Vista/2008 guests)? I''m not sure the shim provides locking enhancements, but we have seen benefits for certain workloads. Are you running PV on HVM drivers? One last area of concern may be distributing callback interrupts to all VCPUS. We still run Xen (SLES10 SP2 based) with a patch to force all guest callback interrupts to VCPU 0. This has consistently improved our SMP guest I/O performance while running PV-on-HVM drivers. Steve> Federico > -----Original Message----- > From: Keir Fraser [mailto:keir.fraser@eu.citrix.com] > Sent: Wednesday, December 31, 2008 5:10 AM > To: Andrew Lyon > Cc: Venefax; James Harper; Dirk Utterback; xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] Windows SMP > > On 31/12/2008 10:08, "Andrew Lyon" <andrew.lyon@gmail.com> wrote: > >> Thanks Keir, I added a couple of printk''s as you suggested and I can >> now see if the feature is supported: >> >> Intel(R) Xeon(R) CPU E5420 @ 2.50GHz = (XEN) APIC Access >> virtualized >> Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz = (XEN) APIC Access > emulated >> I didnt expect my Xeon system to support it, I''ve read some pdf''s >> about Intel virtualizaton features and they seemed to suggest it was a >> new feature on 7xxx Xeons. >> >> The results fit the performance I''ve seen, a windows xp 32 bit hvm >> with 2 cpu''s runs a lot faster on the Xeon 2.5 than on the Core 2.4. > > I think it is probably worth printing out. I''ll add a patch to xen-unstable. > > Thanks, > Keir > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Venefax wrote:> I use only Windows 2003 R2 SP2 fully patched. Therefore that patch would not apply since it is for Vista (Novell shim (Vista/2008 guests). But I will ask Novell. > I would love to try that second patch that would "force all guest callback interrupts to VCPU 0". My setup is to use one core for xen, only, and the rest 15 cores for virtual machines, as instructed by Novell support. I am not over-committing CPU''s. Even if I run one single virtual machine with 8 CPU''s, the performance is awful. It takes 10 mins to load an application that loads in 1.5-2.0 mins with one single CPU. I switched all my 16 virtual machines to Standard PC Hal and that is the only way that the application works. However, it is an administrative nightmare. Instead of managing 16 independent windows machines via VNC, I should be able to manage two, maybe one. > Can you send me information regarding both patches mentioned?I''ll have to check into the state of the PAUSE exit patch. We are doing the work on an Intel Tylersburg machine. I don''t think these are generally available yet. I have attached the referenced callback IRQ routing patch. Whose Windows PV-on-HVM drivers are you running? Steve> Federico > > > -----Original Message----- > From: Steve Ofsthun [mailto:sofsthun@virtualiron.com] > Sent: Monday, January 05, 2009 1:59 PM > To: Venefax > Cc: ''Keir Fraser''; ''Andrew Lyon''; ''James Harper''; ''Dirk Utterback''; xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] Windows SMP > > Venefax wrote: >> Dear Gentlemen > >> I have a machine with Intel 7350, 4 quad core. I can test since I have two >> virtual machines with 8 vcpus, showing a gross overhead that makes them >> unsuited for business. I use SLES 10 SP2, and I don''t know how to apply the >> patch, but if somebody can log in and apply it, we can see the results >> immediately. The issue is affecting me directly. My two VM''s have a VOIP >> application, very intensive in network and CPU usage. > > If your problem isn''t the previously discussed TPR issues ... > > What Windows version are you running in your guests? > > Windows 2000 SMP has serious problems due to the CPU wasting idle loop. We avoid this with a special idler daemon running in the guest. > > > Are you over-committing VCPUS at all (more than one active VCPU per CPU)? > > We have noticed significant performance degradation with SMP windows once you over-commit VCPUS on the host. This seems to be due to excessive guest spinlock overhead caused by the spinning VCPUS wasting their entire quantum looping for the guest lock at the same time they are preventing the guest lock holding VCPU from running. We have been experimenting with new CPU features for exiting on PAUSE instructions. This can possibly be used to detect the CPU wasting spinners. > > > Have you tried using the Novell shim (Vista/2008 guests)? > > I''m not sure the shim provides locking enhancements, but we have seen benefits for certain workloads. > > > Are you running PV on HVM drivers? > > One last area of concern may be distributing callback interrupts to all VCPUS. We still run Xen (SLES10 SP2 based) with a patch to force all guest callback interrupts to VCPU 0. This has consistently improved our SMP guest I/O performance while running PV-on-HVM drivers. > > > Steve > >> Federico >> -----Original Message----- >> From: Keir Fraser [mailto:keir.fraser@eu.citrix.com] >> Sent: Wednesday, December 31, 2008 5:10 AM >> To: Andrew Lyon >> Cc: Venefax; James Harper; Dirk Utterback; xen-devel@lists.xensource.com >> Subject: Re: [Xen-devel] Windows SMP >> >> On 31/12/2008 10:08, "Andrew Lyon" <andrew.lyon@gmail.com> wrote: >> >>> Thanks Keir, I added a couple of printk''s as you suggested and I can >>> now see if the feature is supported: >>> >>> Intel(R) Xeon(R) CPU E5420 @ 2.50GHz = (XEN) APIC Access >>> virtualized >>> Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz = (XEN) APIC Access >> emulated >>> I didnt expect my Xeon system to support it, I''ve read some pdf''s >>> about Intel virtualizaton features and they seemed to suggest it was a >>> new feature on 7xxx Xeons. >>> >>> The results fit the performance I''ve seen, a windows xp 32 bit hvm >>> with 2 cpu''s runs a lot faster on the Xeon 2.5 than on the Core 2.4. >> I think it is probably worth printing out. I''ll add a patch to xen-unstable. >> >> Thanks, >> Keir >> >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
I am using two version of the drivers, with identical limitations. James'' GPLPV and the Novell drivers. Federico -----Original Message----- From: Steve Ofsthun [mailto:sofsthun@virtualiron.com] Sent: Monday, January 05, 2009 3:05 PM To: Venefax Cc: ''Keir Fraser''; ''Andrew Lyon''; ''James Harper''; ''Dirk Utterback''; xen-devel@lists.xensource.com Subject: Re: [Xen-devel] Windows SMP Venefax wrote:> I use only Windows 2003 R2 SP2 fully patched. Therefore that patch would not apply since it is for Vista (Novell shim (Vista/2008 guests). But I will ask Novell. > I would love to try that second patch that would "force all guest callback interrupts to VCPU 0". My setup is to use one core for xen, only, and the rest 15 cores for virtual machines, as instructed by Novell support. I am not over-committing CPU''s. Even if I run one single virtual machine with 8 CPU''s, the performance is awful. It takes 10 mins to load an application that loads in 1.5-2.0 mins with one single CPU. I switched all my 16 virtual machines to Standard PC Hal and that is the only way that the application works. However, it is an administrative nightmare. Instead of managing 16 independent windows machines via VNC, I should be able to manage two, maybe one. > Can you send me information regarding both patches mentioned?I''ll have to check into the state of the PAUSE exit patch. We are doing the work on an Intel Tylersburg machine. I don''t think these are generally available yet. I have attached the referenced callback IRQ routing patch. Whose Windows PV-on-HVM drivers are you running? Steve> Federico > > > -----Original Message----- > From: Steve Ofsthun [mailto:sofsthun@virtualiron.com] > Sent: Monday, January 05, 2009 1:59 PM > To: Venefax > Cc: ''Keir Fraser''; ''Andrew Lyon''; ''James Harper''; ''Dirk Utterback''; > xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] Windows SMP > > Venefax wrote: >> Dear Gentlemen > >> I have a machine with Intel 7350, 4 quad core. I can test since I >> have two virtual machines with 8 vcpus, showing a gross overhead that >> makes them unsuited for business. I use SLES 10 SP2, and I don''t know >> how to apply the patch, but if somebody can log in and apply it, we >> can see the results immediately. The issue is affecting me directly. >> My two VM''s have a VOIP application, very intensive in network and CPU usage. > > If your problem isn''t the previously discussed TPR issues ... > > What Windows version are you running in your guests? > > Windows 2000 SMP has serious problems due to the CPU wasting idle loop. We avoid this with a special idler daemon running in the guest. > > > Are you over-committing VCPUS at all (more than one active VCPU per CPU)? > > We have noticed significant performance degradation with SMP windows once you over-commit VCPUS on the host. This seems to be due to excessive guest spinlock overhead caused by the spinning VCPUS wasting their entire quantum looping for the guest lock at the same time they are preventing the guest lock holding VCPU from running. We have been experimenting with new CPU features for exiting on PAUSE instructions. This can possibly be used to detect the CPU wasting spinners. > > > Have you tried using the Novell shim (Vista/2008 guests)? > > I''m not sure the shim provides locking enhancements, but we have seen benefits for certain workloads. > > > Are you running PV on HVM drivers? > > One last area of concern may be distributing callback interrupts to all VCPUS. We still run Xen (SLES10 SP2 based) with a patch to force all guest callback interrupts to VCPU 0. This has consistently improved our SMP guest I/O performance while running PV-on-HVM drivers. > > > Steve > >> Federico >> -----Original Message----- >> From: Keir Fraser [mailto:keir.fraser@eu.citrix.com] >> Sent: Wednesday, December 31, 2008 5:10 AM >> To: Andrew Lyon >> Cc: Venefax; James Harper; Dirk Utterback; >> xen-devel@lists.xensource.com >> Subject: Re: [Xen-devel] Windows SMP >> >> On 31/12/2008 10:08, "Andrew Lyon" <andrew.lyon@gmail.com> wrote: >> >>> Thanks Keir, I added a couple of printk''s as you suggested and I can >>> now see if the feature is supported: >>> >>> Intel(R) Xeon(R) CPU E5420 @ 2.50GHz = (XEN) APIC Access >>> virtualized >>> Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz = (XEN) APIC Access >> emulated >>> I didnt expect my Xeon system to support it, I''ve read some pdf''s >>> about Intel virtualizaton features and they seemed to suggest it was >>> a new feature on 7xxx Xeons. >>> >>> The results fit the performance I''ve seen, a windows xp 32 bit hvm >>> with 2 cpu''s runs a lot faster on the Xeon 2.5 than on the Core 2.4. >> I think it is probably worth printing out. I''ll add a patch to xen-unstable. >> >> Thanks, >> Keir >> >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Dear Gentlemen One silly question, maybe, how do I apply the patch supplied in SLES SP2? I have not compiled anything from sources. Federico _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Venefax wrote:> Dear Gentlemen > One silly question, maybe, how do I apply the patch supplied in SLES SP2? I have not compiled anything from sources. > FedericoI believe the proper sequence is: download the SLES10 SDK iso from you Novell support account: SLE-10-SP2-SDK-DVD-x86_64-GM-DVD1.iso Add this ISO as a yast installation source. install the xen-3.2.0_16718_14-0.4.src.rpm rpm via yast or: # rpm -i xen-3.2.0_16718_14-0.4.src.rpm Note that this is the .src.rpm not the binary rpm. Prep the source tree using: # rpmbuild -bc /usr/src/packages/SPECS/xen.spec This should build, patch and compile the package. If it fails due to dependencies, add the missing RPMs using yast. Once it succeeds, apply the attached patch (I attached the wrong one previously): # cd /usr/src/packages/BUILD/xen-3.2-testing # patch -p0 < ~/xen-vioapic-callback-routing.patch Build the hypervisor: # cd /usr/src/packages/BUILD/xen-3.2-testing # make xen try out the resulting xen/xen.gz manually. Steve _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Dear Steve I went as far as "make xen". The question is now what do I do to use this new, patched hypervisor, I mean, how do I put in production? A simple reboot? Please advise. Federico -----Original Message----- From: Steve Ofsthun [mailto:sofsthun@virtualiron.com] Sent: Tuesday, January 06, 2009 4:20 PM To: Venefax Cc: ''Keir Fraser''; ''Andrew Lyon''; ''James Harper''; ''Dirk Utterback''; xen-devel@lists.xensource.com Subject: Re: [Xen-devel] Windows SMP Venefax wrote:> Dear Gentlemen > One silly question, maybe, how do I apply the patch supplied in SLES SP2? I have not compiled anything from sources. > FedericoI believe the proper sequence is: download the SLES10 SDK iso from you Novell support account: SLE-10-SP2-SDK-DVD-x86_64-GM-DVD1.iso Add this ISO as a yast installation source. install the xen-3.2.0_16718_14-0.4.src.rpm rpm via yast or: # rpm -i xen-3.2.0_16718_14-0.4.src.rpm Note that this is the .src.rpm not the binary rpm. Prep the source tree using: # rpmbuild -bc /usr/src/packages/SPECS/xen.spec This should build, patch and compile the package. If it fails due to dependencies, add the missing RPMs using yast. Once it succeeds, apply the attached patch (I attached the wrong one previously): # cd /usr/src/packages/BUILD/xen-3.2-testing # patch -p0 < ~/xen-vioapic-callback-routing.patch Build the hypervisor: # cd /usr/src/packages/BUILD/xen-3.2-testing # make xen try out the resulting xen/xen.gz manually. Steve _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Venefax wrote:> I went as far as "make xen". The question is now what do I do to use this new, patched hypervisor, I mean, how do I put in production? A simple reboot? Please advise.As we don''t want to assume that this test xen will even boot, the safest thing is to edit your /boot/grub/menu.lst (or use yast [System -> Boot Loader]) to copy your current xen boot entry. Leave everything identical between the two entries except for the hypervisor (change /boot/xen.gz to /boot/xen-test.gz) and the title line to indicate the test entry (add something like "- test" to the end of the title line). If you use yast, the title entry is in the label column. Copy your newly created xen.gz to /boot/xen-test.gz (be careful not to copy your new file on top of any existing /boot/xen.gz). Reboot the node and manually select the test selection. If the boot succeeds, try your experiment. If the boot fails, just reboot normally and verify your changes to the menu.lst.> Federico > > -----Original Message----- > From: Steve Ofsthun [mailto:sofsthun@virtualiron.com] > Sent: Tuesday, January 06, 2009 4:20 PM > To: Venefax > Cc: ''Keir Fraser''; ''Andrew Lyon''; ''James Harper''; ''Dirk Utterback''; xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] Windows SMP > > Venefax wrote: >> Dear Gentlemen >> One silly question, maybe, how do I apply the patch supplied in SLES SP2? I have not compiled anything from sources. >> Federico > > I believe the proper sequence is: > > download the SLES10 SDK iso from you Novell support account: > > SLE-10-SP2-SDK-DVD-x86_64-GM-DVD1.iso > > Add this ISO as a yast installation source. > > install the xen-3.2.0_16718_14-0.4.src.rpm rpm via yast or: > > # rpm -i xen-3.2.0_16718_14-0.4.src.rpm > > Note that this is the .src.rpm not the binary rpm. > > Prep the source tree using: > > # rpmbuild -bc /usr/src/packages/SPECS/xen.spec > > This should build, patch and compile the package. If it fails due to dependencies, add the missing RPMs using yast. > > Once it succeeds, apply the attached patch (I attached the wrong one previously): > > # cd /usr/src/packages/BUILD/xen-3.2-testing > # patch -p0 < ~/xen-vioapic-callback-routing.patch > > Build the hypervisor: > > # cd /usr/src/packages/BUILD/xen-3.2-testing > # make xen > > try out the resulting xen/xen.gz manually. > > Steve > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
It booted fine, but then I found out that I actually went back in my version. Please let me know if I should go ahead or not. I downloaded the source file that you instructed me, xen-3.2.0_16718_14-0.4.src.rpm, but it turns out that I was using 3.2.0_16718_18-0.3.src.rpm, which I could not find in Google or Novell. I applied the patch and after rebooting I found that I am of course using _14-0.4 instead of 18-0.3. So the question is, does it matter? Or is it better to use this lower version with the patch? If it matters indeed, how do I get the _18.03 source rpm? The other question is: is it possible to update the whole xen + tools to 3.3.1rc4 in Suse 10 SP2 and is this patch still valid in that scenario? Federico -----Original Message----- From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Steve Ofsthun Sent: Wednesday, January 07, 2009 9:47 AM To: Venefax Cc: xen-devel@lists.xensource.com; ''Andrew Lyon''; ''Dirk Utterback''; ''Keir Fraser''; ''James Harper'' Subject: Re: [Xen-devel] Windows SMP Venefax wrote:> I went as far as "make xen". The question is now what do I do to use this new, patched hypervisor, I mean, how do I put in production? A simple reboot? Please advise.As we don''t want to assume that this test xen will even boot, the safest thing is to edit your /boot/grub/menu.lst (or use yast [System -> Boot Loader]) to copy your current xen boot entry. Leave everything identical between the two entries except for the hypervisor (change /boot/xen.gz to /boot/xen-test.gz) and the title line to indicate the test entry (add something like "- test" to the end of the title line). If you use yast, the title entry is in the label column. Copy your newly created xen.gz to /boot/xen-test.gz (be careful not to copy your new file on top of any existing /boot/xen.gz). Reboot the node and manually select the test selection. If the boot succeeds, try your experiment. If the boot fails, just reboot normally and verify your changes to the menu.lst.> Federico > > -----Original Message----- > From: Steve Ofsthun [mailto:sofsthun@virtualiron.com] > Sent: Tuesday, January 06, 2009 4:20 PM > To: Venefax > Cc: ''Keir Fraser''; ''Andrew Lyon''; ''James Harper''; ''Dirk Utterback''; xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] Windows SMP > > Venefax wrote: >> Dear Gentlemen >> One silly question, maybe, how do I apply the patch supplied in SLES SP2? I have not compiled anything from sources. >> Federico > > I believe the proper sequence is: > > download the SLES10 SDK iso from you Novell support account: > > SLE-10-SP2-SDK-DVD-x86_64-GM-DVD1.iso > > Add this ISO as a yast installation source. > > install the xen-3.2.0_16718_14-0.4.src.rpm rpm via yast or: > > # rpm -i xen-3.2.0_16718_14-0.4.src.rpm > > Note that this is the .src.rpm not the binary rpm. > > Prep the source tree using: > > # rpmbuild -bc /usr/src/packages/SPECS/xen.spec > > This should build, patch and compile the package. If it fails due to dependencies, add the missing RPMs using yast. > > Once it succeeds, apply the attached patch (I attached the wrong one previously): > > # cd /usr/src/packages/BUILD/xen-3.2-testing > # patch -p0 < ~/xen-vioapic-callback-routing.patch > > Build the hypervisor: > > # cd /usr/src/packages/BUILD/xen-3.2-testing > # make xen > > try out the resulting xen/xen.gz manually. > > Steve > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Venefax wrote:> It booted fine, but then I found out that I actually went back in my version. Please let me know if I should go ahead or not. I downloaded the source file that you instructed me, xen-3.2.0_16718_14-0.4.src.rpm, but it turns out that I was using 3.2.0_16718_18-0.3.src.rpm, which I could not find in Google or Novell. I applied the patch and after rebooting I found that I am of course using _14-0.4 instead of 18-0.3. So the question is, does it matter? Or is it better to use this lower version with the patch? If it matters indeed, how do I get the _18.03 source rpm?Obviously running the correct version would be best. I''m guessing my version is the original shipping SP2 not the latest SP2 kernel/xen update. You should be able to get the proper src.rpm from the same place you got the binary rpm from.> The other question is: is it possible to update the whole xen + tools to 3.3.1rc4 in Suse 10 SP2 and is this patch still valid in that scenario?The short answer is no. This is a much bigger job. The dom0 kernel is intimately tied to the underlying Xen version. If you want to try a newer Xen, you would have to move to OpenSUSE 11.1 or gain access to the SLES11 Beta. As far as the supplied patch, it should apply since the underlying code is still present in xen-unstable. Steve> Federico > > > -----Original Message----- > From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Steve Ofsthun > Sent: Wednesday, January 07, 2009 9:47 AM > To: Venefax > Cc: xen-devel@lists.xensource.com; ''Andrew Lyon''; ''Dirk Utterback''; ''Keir Fraser''; ''James Harper'' > Subject: Re: [Xen-devel] Windows SMP > > Venefax wrote: > >> I went as far as "make xen". The question is now what do I do to use this new, patched hypervisor, I mean, how do I put in production? A simple reboot? Please advise. > > As we don''t want to assume that this test xen will even boot, the safest thing is to edit your /boot/grub/menu.lst (or use yast [System -> Boot Loader]) to copy your current xen boot entry. Leave everything identical between the two entries except for the hypervisor (change /boot/xen.gz to /boot/xen-test.gz) and the title line to indicate the test entry (add something like "- test" to the end of the title line). If you use yast, the title entry is in the label column. > > Copy your newly created xen.gz to /boot/xen-test.gz (be careful not to copy your new file on top of any existing /boot/xen.gz). > > Reboot the node and manually select the test selection. If the boot succeeds, try your experiment. If the boot fails, just reboot normally and verify your changes to the menu.lst. > >> Federico >> >> -----Original Message----- >> From: Steve Ofsthun [mailto:sofsthun@virtualiron.com] >> Sent: Tuesday, January 06, 2009 4:20 PM >> To: Venefax >> Cc: ''Keir Fraser''; ''Andrew Lyon''; ''James Harper''; ''Dirk Utterback''; xen-devel@lists.xensource.com >> Subject: Re: [Xen-devel] Windows SMP >> >> Venefax wrote: >>> Dear Gentlemen >>> One silly question, maybe, how do I apply the patch supplied in SLES SP2? I have not compiled anything from sources. >>> Federico >> I believe the proper sequence is: >> >> download the SLES10 SDK iso from you Novell support account: >> >> SLE-10-SP2-SDK-DVD-x86_64-GM-DVD1.iso >> >> Add this ISO as a yast installation source. >> >> install the xen-3.2.0_16718_14-0.4.src.rpm rpm via yast or: >> >> # rpm -i xen-3.2.0_16718_14-0.4.src.rpm >> >> Note that this is the .src.rpm not the binary rpm. >> >> Prep the source tree using: >> >> # rpmbuild -bc /usr/src/packages/SPECS/xen.spec >> >> This should build, patch and compile the package. If it fails due to dependencies, add the missing RPMs using yast. >> >> Once it succeeds, apply the attached patch (I attached the wrong one previously): >> >> # cd /usr/src/packages/BUILD/xen-3.2-testing >> # patch -p0 < ~/xen-vioapic-callback-routing.patch >> >> Build the hypervisor: >> >> # cd /usr/src/packages/BUILD/xen-3.2-testing >> # make xen >> >> try out the resulting xen/xen.gz manually. >> >> Steve >> >> >> > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
I put the patch in production. Is there any way to know if it is working. I cannot observe any effect, either positive or negative. I have 16 windows VM with one virtual processor each, the Standard PC Hal, and an application that can bring the cpu usage to 100%. Heavy network usage. It is a telephony application. Does anybody have any other patch that would make it use less CPU? Federico -----Original Message----- From: Steve Ofsthun [mailto:sofsthun@virtualiron.com] Sent: Wednesday, January 07, 2009 10:42 AM To: Venefax Cc: xen-devel@lists.xensource.com; ''Andrew Lyon''; ''Dirk Utterback''; ''Keir Fraser'' Subject: Re: [Xen-devel] Windows SMP Venefax wrote:> It booted fine, but then I found out that I actually went back in my version. Please let me know if I should go ahead or not. I downloaded the source file that you instructed me, xen-3.2.0_16718_14-0.4.src.rpm, but it turns out that I was using 3.2.0_16718_18-0.3.src.rpm, which I could not find in Google or Novell. I applied the patch and after rebooting I found that I am of course using _14-0.4 instead of 18-0.3. So the question is, does it matter? Or is it better to use this lower version with the patch? If it matters indeed, how do I get the _18.03 source rpm?Obviously running the correct version would be best. I''m guessing my version is the original shipping SP2 not the latest SP2 kernel/xen update. You should be able to get the proper src.rpm from the same place you got the binary rpm from.> The other question is: is it possible to update the whole xen + tools to 3.3.1rc4 in Suse 10 SP2 and is this patch still valid in that scenario?The short answer is no. This is a much bigger job. The dom0 kernel is intimately tied to the underlying Xen version. If you want to try a newer Xen, you would have to move to OpenSUSE 11.1 or gain access to the SLES11 Beta. As far as the supplied patch, it should apply since the underlying code is still present in xen-unstable. Steve> Federico > > > -----Original Message----- > From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Steve Ofsthun > Sent: Wednesday, January 07, 2009 9:47 AM > To: Venefax > Cc: xen-devel@lists.xensource.com; ''Andrew Lyon''; ''Dirk Utterback''; ''Keir Fraser''; ''James Harper'' > Subject: Re: [Xen-devel] Windows SMP > > Venefax wrote: > >> I went as far as "make xen". The question is now what do I do to use this new, patched hypervisor, I mean, how do I put in production? A simple reboot? Please advise. > > As we don''t want to assume that this test xen will even boot, the safest thing is to edit your /boot/grub/menu.lst (or use yast [System -> Boot Loader]) to copy your current xen boot entry. Leave everything identical between the two entries except for the hypervisor (change /boot/xen.gz to /boot/xen-test.gz) and the title line to indicate the test entry (add something like "- test" to the end of the title line). If you use yast, the title entry is in the label column. > > Copy your newly created xen.gz to /boot/xen-test.gz (be careful not to copy your new file on top of any existing /boot/xen.gz). > > Reboot the node and manually select the test selection. If the boot succeeds, try your experiment. If the boot fails, just reboot normally and verify your changes to the menu.lst. > >> Federico >> >> -----Original Message----- >> From: Steve Ofsthun [mailto:sofsthun@virtualiron.com] >> Sent: Tuesday, January 06, 2009 4:20 PM >> To: Venefax >> Cc: ''Keir Fraser''; ''Andrew Lyon''; ''James Harper''; ''Dirk Utterback''; xen-devel@lists.xensource.com >> Subject: Re: [Xen-devel] Windows SMP >> >> Venefax wrote: >>> Dear Gentlemen >>> One silly question, maybe, how do I apply the patch supplied in SLES SP2? I have not compiled anything from sources. >>> Federico >> I believe the proper sequence is: >> >> download the SLES10 SDK iso from you Novell support account: >> >> SLE-10-SP2-SDK-DVD-x86_64-GM-DVD1.iso >> >> Add this ISO as a yast installation source. >> >> install the xen-3.2.0_16718_14-0.4.src.rpm rpm via yast or: >> >> # rpm -i xen-3.2.0_16718_14-0.4.src.rpm >> >> Note that this is the .src.rpm not the binary rpm. >> >> Prep the source tree using: >> >> # rpmbuild -bc /usr/src/packages/SPECS/xen.spec >> >> This should build, patch and compile the package. If it fails due to dependencies, add the missing RPMs using yast. >> >> Once it succeeds, apply the attached patch (I attached the wrong one previously): >> >> # cd /usr/src/packages/BUILD/xen-3.2-testing >> # patch -p0 < ~/xen-vioapic-callback-routing.patch >> >> Build the hypervisor: >> >> # cd /usr/src/packages/BUILD/xen-3.2-testing >> # make xen >> >> try out the resulting xen/xen.gz manually. >> >> Steve >> >> >> > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Venefax wrote:> I put the patch in production. Is there any way to know if it is working. I cannot observe any effect, either positive or negative. I have 16 windows VM with one virtual processor each, the Standard PC Hal, and an application that can bring the cpu usage to 100%. Heavy network usage. It is a telephony application. Does anybody have any other patch that would make it use less CPU?Now I am confused. I thought your original problem was related to guests with multiple vcpus (as the subject implies). The patch addresses additional latency added when more than 1 vcpu is involved in the interrupt delivery. The patch will have no effect on 1-vcpu guests. With only 1 vcpu, interrupts can only be delivered to one place already (vcpu 0). Steve> Federico > > -----Original Message----- > From: Steve Ofsthun [mailto:sofsthun@virtualiron.com] > Sent: Wednesday, January 07, 2009 10:42 AM > To: Venefax > Cc: xen-devel@lists.xensource.com; ''Andrew Lyon''; ''Dirk Utterback''; ''Keir Fraser'' > Subject: Re: [Xen-devel] Windows SMP > > Venefax wrote: >> It booted fine, but then I found out that I actually went back in my version. Please let me know if I should go ahead or not. I downloaded the source file that you instructed me, xen-3.2.0_16718_14-0.4.src.rpm, but it turns out that I was using 3.2.0_16718_18-0.3.src.rpm, which I could not find in Google or Novell. I applied the patch and after rebooting I found that I am of course using _14-0.4 instead of 18-0.3. So the question is, does it matter? Or is it better to use this lower version with the patch? If it matters indeed, how do I get the _18.03 source rpm? > > Obviously running the correct version would be best. I''m guessing my version is the original shipping SP2 not the latest SP2 kernel/xen update. You should be able to get the proper src.rpm from the same place you got the binary rpm from. > >> The other question is: is it possible to update the whole xen + tools to 3.3.1rc4 in Suse 10 SP2 and is this patch still valid in that scenario? > > The short answer is no. This is a much bigger job. The dom0 kernel is intimately tied to the underlying Xen version. If you want to try a newer Xen, you would have to move to OpenSUSE 11.1 or gain access to the SLES11 Beta. > > As far as the supplied patch, it should apply since the underlying code is still present in xen-unstable. > > Steve > >> Federico >> >> >> -----Original Message----- >> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Steve Ofsthun >> Sent: Wednesday, January 07, 2009 9:47 AM >> To: Venefax >> Cc: xen-devel@lists.xensource.com; ''Andrew Lyon''; ''Dirk Utterback''; ''Keir Fraser''; ''James Harper'' >> Subject: Re: [Xen-devel] Windows SMP >> >> Venefax wrote: >> >>> I went as far as "make xen". The question is now what do I do to use this new, patched hypervisor, I mean, how do I put in production? A simple reboot? Please advise. >> As we don''t want to assume that this test xen will even boot, the safest thing is to edit your /boot/grub/menu.lst (or use yast [System -> Boot Loader]) to copy your current xen boot entry. Leave everything identical between the two entries except for the hypervisor (change /boot/xen.gz to /boot/xen-test.gz) and the title line to indicate the test entry (add something like "- test" to the end of the title line). If you use yast, the title entry is in the label column. >> >> Copy your newly created xen.gz to /boot/xen-test.gz (be careful not to copy your new file on top of any existing /boot/xen.gz). >> >> Reboot the node and manually select the test selection. If the boot succeeds, try your experiment. If the boot fails, just reboot normally and verify your changes to the menu.lst. >> >>> Federico >>> >>> -----Original Message----- >>> From: Steve Ofsthun [mailto:sofsthun@virtualiron.com] >>> Sent: Tuesday, January 06, 2009 4:20 PM >>> To: Venefax >>> Cc: ''Keir Fraser''; ''Andrew Lyon''; ''James Harper''; ''Dirk Utterback''; xen-devel@lists.xensource.com >>> Subject: Re: [Xen-devel] Windows SMP >>> >>> Venefax wrote: >>>> Dear Gentlemen >>>> One silly question, maybe, how do I apply the patch supplied in SLES SP2? I have not compiled anything from sources. >>>> Federico >>> I believe the proper sequence is: >>> >>> download the SLES10 SDK iso from you Novell support account: >>> >>> SLE-10-SP2-SDK-DVD-x86_64-GM-DVD1.iso >>> >>> Add this ISO as a yast installation source. >>> >>> install the xen-3.2.0_16718_14-0.4.src.rpm rpm via yast or: >>> >>> # rpm -i xen-3.2.0_16718_14-0.4.src.rpm >>> >>> Note that this is the .src.rpm not the binary rpm. >>> >>> Prep the source tree using: >>> >>> # rpmbuild -bc /usr/src/packages/SPECS/xen.spec >>> >>> This should build, patch and compile the package. If it fails due to dependencies, add the missing RPMs using yast. >>> >>> Once it succeeds, apply the attached patch (I attached the wrong one previously): >>> >>> # cd /usr/src/packages/BUILD/xen-3.2-testing >>> # patch -p0 < ~/xen-vioapic-callback-routing.patch >>> >>> Build the hypervisor: >>> >>> # cd /usr/src/packages/BUILD/xen-3.2-testing >>> # make xen >>> >>> try out the resulting xen/xen.gz manually. >>> >>> Steve >>> >>> >>> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Dear Steve I am trying to apply the xen-vioapic-callback-routing.patch to xen 3.3 as supplied by Suse 11.1. The question is: is it necessary? I use both SMP (8 cores) para-virtualized Centos 5.2 and fully-virtualized Windows SMP (4 cores). Do you recommend that I apply this patch? Or the patch is already in Xen 3.3? Yours Federico -----Original Message----- From: Steve Ofsthun [mailto:sofsthun@virtualiron.com] Sent: Friday, January 09, 2009 9:30 AM To: Venefax Cc: xen-devel@lists.xensource.com; ''Andrew Lyon''; ''Dirk Utterback''; ''Keir Fraser'' Subject: Re: [Xen-devel] Windows SMP Venefax wrote:> I put the patch in production. Is there any way to know if it is working. I cannot observe any effect, either positive or negative. I have 16 windows VM with one virtual processor each, the Standard PC Hal, and an application that can bring the cpu usage to 100%. Heavy network usage. It is a telephony application. Does anybody have any other patch that would make it use less CPU?Now I am confused. I thought your original problem was related to guests with multiple vcpus (as the subject implies). The patch addresses additional latency added when more than 1 vcpu is involved in the interrupt delivery. The patch will have no effect on 1-vcpu guests. With only 1 vcpu, interrupts can only be delivered to one place already (vcpu 0). Steve> Federico > > -----Original Message----- > From: Steve Ofsthun [mailto:sofsthun@virtualiron.com] > Sent: Wednesday, January 07, 2009 10:42 AM > To: Venefax > Cc: xen-devel@lists.xensource.com; ''Andrew Lyon''; ''Dirk Utterback''; ''Keir Fraser'' > Subject: Re: [Xen-devel] Windows SMP > > Venefax wrote: >> It booted fine, but then I found out that I actually went back in my version. Please let me know if I should go ahead or not. I downloaded the source file that you instructed me, xen-3.2.0_16718_14-0.4.src.rpm, but it turns out that I was using 3.2.0_16718_18-0.3.src.rpm, which I could not find in Google or Novell. I applied the patch and after rebooting I found that I am of course using _14-0.4 instead of 18-0.3. So the question is, does it matter? Or is it better to use this lower version with the patch? If it matters indeed, how do I get the _18.03 source rpm? > > Obviously running the correct version would be best. I''m guessing my version is the original shipping SP2 not the latest SP2 kernel/xen update. You should be able to get the proper src.rpm from the same place you got the binary rpm from. > >> The other question is: is it possible to update the whole xen + tools to 3.3.1rc4 in Suse 10 SP2 and is this patch still valid in that scenario? > > The short answer is no. This is a much bigger job. The dom0 kernel is intimately tied to the underlying Xen version. If you want to try a newer Xen, you would have to move to OpenSUSE 11.1 or gain access to the SLES11 Beta. > > As far as the supplied patch, it should apply since the underlying code is still present in xen-unstable. > > Steve > >> Federico >> >> >> -----Original Message----- >> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Steve Ofsthun >> Sent: Wednesday, January 07, 2009 9:47 AM >> To: Venefax >> Cc: xen-devel@lists.xensource.com; ''Andrew Lyon''; ''Dirk Utterback''; ''Keir Fraser''; ''James Harper'' >> Subject: Re: [Xen-devel] Windows SMP >> >> Venefax wrote: >> >>> I went as far as "make xen". The question is now what do I do to use this new, patched hypervisor, I mean, how do I put in production? A simple reboot? Please advise. >> As we don''t want to assume that this test xen will even boot, the safest thing is to edit your /boot/grub/menu.lst (or use yast [System -> Boot Loader]) to copy your current xen boot entry. Leave everything identical between the two entries except for the hypervisor (change /boot/xen.gz to /boot/xen-test.gz) and the title line to indicate the test entry (add something like "- test" to the end of the title line). If you use yast, the title entry is in the label column. >> >> Copy your newly created xen.gz to /boot/xen-test.gz (be careful not to copy your new file on top of any existing /boot/xen.gz). >> >> Reboot the node and manually select the test selection. If the boot succeeds, try your experiment. If the boot fails, just reboot normally and verify your changes to the menu.lst. >> >>> Federico >>> >>> -----Original Message----- >>> From: Steve Ofsthun [mailto:sofsthun@virtualiron.com] >>> Sent: Tuesday, January 06, 2009 4:20 PM >>> To: Venefax >>> Cc: ''Keir Fraser''; ''Andrew Lyon''; ''James Harper''; ''Dirk Utterback''; xen-devel@lists.xensource.com >>> Subject: Re: [Xen-devel] Windows SMP >>> >>> Venefax wrote: >>>> Dear Gentlemen >>>> One silly question, maybe, how do I apply the patch supplied in SLES SP2? I have not compiled anything from sources. >>>> Federico >>> I believe the proper sequence is: >>> >>> download the SLES10 SDK iso from you Novell support account: >>> >>> SLE-10-SP2-SDK-DVD-x86_64-GM-DVD1.iso >>> >>> Add this ISO as a yast installation source. >>> >>> install the xen-3.2.0_16718_14-0.4.src.rpm rpm via yast or: >>> >>> # rpm -i xen-3.2.0_16718_14-0.4.src.rpm >>> >>> Note that this is the .src.rpm not the binary rpm. >>> >>> Prep the source tree using: >>> >>> # rpmbuild -bc /usr/src/packages/SPECS/xen.spec >>> >>> This should build, patch and compile the package. If it fails due to dependencies, add the missing RPMs using yast. >>> >>> Once it succeeds, apply the attached patch (I attached the wrong one previously): >>> >>> # cd /usr/src/packages/BUILD/xen-3.2-testing >>> # patch -p0 < ~/xen-vioapic-callback-routing.patch >>> >>> Build the hypervisor: >>> >>> # cd /usr/src/packages/BUILD/xen-3.2-testing >>> # make xen >>> >>> try out the resulting xen/xen.gz manually. >>> >>> Steve >>> >>> >>> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel