Mukesh Rathor
2010-Jun-09 23:09 UTC
[Xen-devel] [PATCHEs]: support more than 32 VCPUs in guests
Hi,
I am attaching three patches, in case anyone is interested. These
patches allow linux guests to support more than 32 VCPUs in 64bit mode
only to whatever linux supports. I tested all 3 with 128 vcpus on a
system with 128 CPU threads. Some scalability work is needed at 128
vcpus (some soft lockups during load) as expected.
Jeremy, pv ops is OK as it is on 128 vcpus, but I reworked the
xen_vcpu_setup() a little to address more than 32vcpus on xen that
doesn''t have vcpu placement. Please take a look.
1. Patch for 5u5 (2.6.18-190*): tested 64bit. compiled 32bit.
2. Patch for 5u4 (2.6.18-164*): tested 64bit. not compiled on 32bit.
(NOTE: increased NR_DYNIRQS to 1024)
3. Patch for PVOPS: minor change to xen_vcpu_setup(). tested 64bit.
not compiled on 32bit.
thanks,
Mukesh
PS: make sure to do full build: vmlinux, modules, etc.. when using el5
kernel patches.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-Jun-09 23:44 UTC
[Xen-devel] Re: [PATCHEs]: support more than 32 VCPUs in guests
On 06/09/2010 04:09 PM, Mukesh Rathor wrote:> Jeremy, pv ops is OK as it is on 128 vcpus, but I reworked the > xen_vcpu_setup() a little to address more than 32vcpus on xen that > doesn''t have vcpu placement. Please take a look. >Why BUG_ON if the number of cpus is too high? Why not just ignore the excess ones? J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Mukesh Rathor
2010-Jun-10 00:08 UTC
[Xen-devel] Re: [PATCHEs]: support more than 32 VCPUs in guests
On Wed, 09 Jun 2010 16:44:02 -0700 Jeremy Fitzhardinge <jeremy@goop.org> wrote:> On 06/09/2010 04:09 PM, Mukesh Rathor wrote: > > Jeremy, pv ops is OK as it is on 128 vcpus, but I reworked the > > xen_vcpu_setup() a little to address more than 32vcpus on xen that > > doesn''t have vcpu placement. Please take a look. > > > > Why BUG_ON if the number of cpus is too high? Why not just ignore the > excess ones? > > JYeah, that was my first thought also... but then i realized i couldn''t just ignore the excess cpus in that function, but would need to go back and fixup all the cpu_present, cpu_online, etc maps (and any assoc data structs, if any), and it just didn''t seem worth it in the 2.6.18* kernels at least. Would have been easier to do if the vcpu setup function returned a value instead of being void. The 2.6.18 kernel will BUG_ON() somewhere right now with excess cpus anyways, so it is not a regression in that sense :)... thanks, Mukesh _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-Jun-10 00:49 UTC
[Xen-devel] Re: [PATCHEs]: support more than 32 VCPUs in guests
On 06/09/2010 05:08 PM, Mukesh Rathor wrote:>> Why BUG_ON if the number of cpus is too high? Why not just ignore the >> excess ones? >> > Yeah, that was my first thought also... but then i realized i couldn''t > just ignore the excess cpus in that function, but would need to go back > and fixup all the cpu_present, cpu_online, etc maps (and any assoc data > structs, if any), and it just didn''t seem worth it in the 2.6.18* > kernels at least. Would have been easier to do if the vcpu setup > function returned a value instead of being void. >Yes, but if have_vcpu_info_placement ends up being false (which is tested before any other cpus are brought up) then you can simply fail to online the ones above the limit. BUG_ON is way too brutal. You need to fail more softly. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Mukesh Rathor
2010-Jun-10 02:13 UTC
[Xen-devel] Re: [PATCHEs]: support more than 32 VCPUs in guests
On Wed, 09 Jun 2010 17:49:52 -0700 Jeremy Fitzhardinge <jeremy@goop.org> wrote:> On 06/09/2010 05:08 PM, Mukesh Rathor wrote: > >> Why BUG_ON if the number of cpus is too high? Why not just ignore > >> the excess ones? > >> > > Yeah, that was my first thought also... but then i realized i > > couldn''t just ignore the excess cpus in that function, but would > > need to go back and fixup all the cpu_present, cpu_online, etc maps > > (and any assoc data structs, if any), and it just didn''t seem worth > > it in the 2.6.18* kernels at least. Would have been easier to do if > > the vcpu setup function returned a value instead of being void. > > > > Yes, but if have_vcpu_info_placement ends up being false (which is > tested before any other cpus are brought up) then you can simply fail > to online the ones above the limit. > > BUG_ON is way too brutal. You need to fail more softly. > > JWell, BUG_ON is only triggered if booting more than 32 VCPUs on a *very old* xen (pre xen 3.1.0). Looking at code closely, we could just set setup_max_cpus to 32 some where in xen function, perhaps even in xen_vcpu_setup(). That way later in smp_init() it would just be ok. One thing tho, the per cpus areas are already setup at that point, so that would need to be cleaned. BTW, I don''t understand why have_vcpu_info_placement is set to 0 in xen_guest_init()? What minimum version of xen is required to run pvops kernel? thanks, Mukesh _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-Jun-14 09:37 UTC
[Xen-devel] Re: [PATCHEs]: support more than 32 VCPUs in guests
On 06/10/2010 03:13 AM, Mukesh Rathor wrote:> Well, BUG_ON is only triggered if booting more than 32 VCPUs on a *very > old* xen (pre xen 3.1.0). > > Looking at code closely, we could just set setup_max_cpus to 32 some > where in xen function, perhaps even in xen_vcpu_setup(). That way > later in smp_init() it would just be ok. >Yes.> One thing tho, the per cpus areas are already setup at that point, so that > would need to be cleaned. BTW, I don''t understand why > have_vcpu_info_placement is set to 0 in xen_guest_init()? >xen_guest_init is used by the pvhvm path, and hvm domains don''t have a notion of vcpu info placement.> What minimum version of xen is required to run pvops kernel? >In theory it should be back-compatible for all Xen 3, but in practice it tweaks lots of bugs in older Xens (particularly 32-on-64). I don''t know that anyone has definitively established an earliest version. I implemented vcpu info placement for use in pvops kernels, but it was never my intention that it be an absolute requirement. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Mukesh Rathor
2010-Jun-15 02:49 UTC
[Xen-devel] Re: [PATCHEs]: support more than 32 VCPUs in guests
On Mon, 14 Jun 2010 10:37:30 +0100 Jeremy Fitzhardinge <jeremy@goop.org> wrote:> On 06/10/2010 03:13 AM, Mukesh Rathor wrote: > > Well, BUG_ON is only triggered if booting more than 32 VCPUs on a > > *very old* xen (pre xen 3.1.0). > > > > Looking at code closely, we could just set setup_max_cpus to 32 some > > where in xen function, perhaps even in xen_vcpu_setup(). That way > > later in smp_init() it would just be ok. > > > > Yes. > > > One thing tho, the per cpus areas are already setup at that point, > > so that would need to be cleaned. BTW, I don''t understand why > > have_vcpu_info_placement is set to 0 in xen_guest_init()? > > > > xen_guest_init is used by the pvhvm path, and hvm domains don''t have a > notion of vcpu info placement. > > > What minimum version of xen is required to run pvops kernel? > > > > In theory it should be back-compatible for all Xen 3, but in practice > it tweaks lots of bugs in older Xens (particularly 32-on-64). I > don''t know that anyone has definitively established an earliest > version. I implemented vcpu info placement for use in pvops kernels, > but it was never my intention that it be an absolute requirement. > > JOk, attached patch without BUG_ON. Please feel free to modify to your liking also. Signed-off-by: Mukesh Rathor <mukesh.rathor@oracle.com> thanks, Mukesh _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Jun-15 05:02 UTC
Re: [Xen-devel] Re: [PATCHEs]: support more than 32 VCPUs in guests
> - struct vcpu_register_vcpu_info info; > - int err; > - struct vcpu_info *vcpup; > -Why the tab to space conversion?> + struct vcpu_register_vcpu_info info; > + int err; > + struct vcpu_info *vcpup;_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-Jun-15 08:30 UTC
[Xen-devel] Re: [PATCHEs]: support more than 32 VCPUs in guests
On 06/15/2010 03:49 AM, Mukesh Rathor wrote:> On Mon, 14 Jun 2010 10:37:30 +0100 > Jeremy Fitzhardinge <jeremy@goop.org> wrote: > > >> On 06/10/2010 03:13 AM, Mukesh Rathor wrote: >> >>> Well, BUG_ON is only triggered if booting more than 32 VCPUs on a >>> *very old* xen (pre xen 3.1.0). >>> >>> Looking at code closely, we could just set setup_max_cpus to 32 some >>> where in xen function, perhaps even in xen_vcpu_setup(). That way >>> later in smp_init() it would just be ok. >>> >>> >> Yes. >> >> >>> One thing tho, the per cpus areas are already setup at that point, >>> so that would need to be cleaned. BTW, I don''t understand why >>> have_vcpu_info_placement is set to 0 in xen_guest_init()? >>> >>> >> xen_guest_init is used by the pvhvm path, and hvm domains don''t have a >> notion of vcpu info placement. >> >> >>> What minimum version of xen is required to run pvops kernel? >>> >>> >> In theory it should be back-compatible for all Xen 3, but in practice >> it tweaks lots of bugs in older Xens (particularly 32-on-64). I >> don''t know that anyone has definitively established an earliest >> version. I implemented vcpu info placement for use in pvops kernels, >> but it was never my intention that it be an absolute requirement. >> >> J >> > Ok, attached patch without BUG_ON. Please feel free to modify > to your liking also. >It looks like you smashed all the tabs into spaces so its hard to see what you''ve changed in the diff. I''ll fix it up and give it a look-over. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Mukesh Rathor
2010-Jun-15 18:45 UTC
[Xen-devel] Re: [PATCHEs]: support more than 32 VCPUs in guests
On Tue, 15 Jun 2010 09:30:35 +0100 Jeremy Fitzhardinge <jeremy@goop.org> wrote:> On 06/15/2010 03:49 AM, Mukesh Rathor wrote: > > On Mon, 14 Jun 2010 10:37:30 +0100 > > Jeremy Fitzhardinge <jeremy@goop.org> wrote: > > > > > >> On 06/10/2010 03:13 AM, Mukesh Rathor wrote: > >> > >>> Well, BUG_ON is only triggered if booting more than 32 VCPUs on a > >>> *very old* xen (pre xen 3.1.0). > >>> > >>> Looking at code closely, we could just set setup_max_cpus to 32 > >>> some where in xen function, perhaps even in xen_vcpu_setup(). > >>> That way later in smp_init() it would just be ok. > >>> > >>> > >> Yes. > >> > >> > >>> One thing tho, the per cpus areas are already setup at that point, > >>> so that would need to be cleaned. BTW, I don''t understand why > >>> have_vcpu_info_placement is set to 0 in xen_guest_init()? > >>> > >>> > >> xen_guest_init is used by the pvhvm path, and hvm domains don''t > >> have a notion of vcpu info placement. > >> > >> > >>> What minimum version of xen is required to run pvops kernel? > >>> > >>> > >> In theory it should be back-compatible for all Xen 3, but in > >> practice it tweaks lots of bugs in older Xens (particularly > >> 32-on-64). I don''t know that anyone has definitively established > >> an earliest version. I implemented vcpu info placement for use in > >> pvops kernels, but it was never my intention that it be an > >> absolute requirement. > >> > >> J > >> > > Ok, attached patch without BUG_ON. Please feel free to modify > > to your liking also. > > > > It looks like you smashed all the tabs into spaces so its hard to see > what you''ve changed in the diff. I''ll fix it up and give it a > look-over. > > JSorry, I''ve tabs turned off because patches I submit to other product I work on must be tab free. Anyways, re attached a new one with tabs. thanks again, Mukesh Signed-off-by: Mukesh Rathor <mukesh.rathor@oracle.com> _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Mukesh Rathor
2010-Jul-17 01:06 UTC
[Xen-devel] Re: [PATCHEs]: support more than 32 VCPUs in guests
On Tue, 15 Jun 2010 11:45:43 -0700 Mukesh Rathor <mukesh.rathor@oracle.com> wrote:> On Tue, 15 Jun 2010 09:30:35 +0100 > Jeremy Fitzhardinge <jeremy@goop.org> wrote: > > > On 06/15/2010 03:49 AM, Mukesh Rathor wrote: > > > On Mon, 14 Jun 2010 10:37:30 +0100 > > > Jeremy Fitzhardinge <jeremy@goop.org> wrote: > > > > > > > > >> On 06/10/2010 03:13 AM, Mukesh Rathor wrote: > > >> > > >>> Well, BUG_ON is only triggered if booting more than 32 VCPUs on > > >>> a *very old* xen (pre xen 3.1.0). > > >>> > > >>> Looking at code closely, we could just set setup_max_cpus to 32 > > >>> some where in xen function, perhaps even in xen_vcpu_setup(). > > >>> That way later in smp_init() it would just be ok. > > >>> > > >>> > > >> Yes. > > >> > > >> > > >>> One thing tho, the per cpus areas are already setup at that > > >>> point, so that would need to be cleaned. BTW, I don''t > > >>> understand why have_vcpu_info_placement is set to 0 in > > >>> xen_guest_init()? > > >>> > > >> xen_guest_init is used by the pvhvm path, and hvm domains don''t > > >> have a notion of vcpu info placement. > > >> > > >> > > >>> What minimum version of xen is required to run pvops kernel? > > >>> > > >>> > > >> In theory it should be back-compatible for all Xen 3, but in > > >> practice it tweaks lots of bugs in older Xens (particularly > > >> 32-on-64). I don''t know that anyone has definitively established > > >> an earliest version. I implemented vcpu info placement for use > > >> in pvops kernels, but it was never my intention that it be an > > >> absolute requirement. > > >> > > >> J > > >> > > > Ok, attached patch without BUG_ON. Please feel free to modify > > > to your liking also. > > > > > > > It looks like you smashed all the tabs into spaces so its hard to > > see what you''ve changed in the diff. I''ll fix it up and give it a > > look-over. > > > > J > > Sorry, I''ve tabs turned off because patches I submit to other product > I work on must be tab free. Anyways, re attached a new one with tabs. > > thanks again, > Mukesh > > Signed-off-by: Mukesh Rathor <mukesh.rathor@oracle.com>Hi Jeremy, Just curious, did this patch ever make it? thanks Mukesh _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-Jul-17 01:09 UTC
[Xen-devel] Re: [PATCHEs]: support more than 32 VCPUs in guests
On 07/16/2010 06:06 PM, Mukesh Rathor wrote:> On Tue, 15 Jun 2010 11:45:43 -0700 > Mukesh Rathor <mukesh.rathor@oracle.com> wrote: > > >> On Tue, 15 Jun 2010 09:30:35 +0100 >> Jeremy Fitzhardinge <jeremy@goop.org> wrote: >> >> >>> On 06/15/2010 03:49 AM, Mukesh Rathor wrote: >>> >>>> On Mon, 14 Jun 2010 10:37:30 +0100 >>>> Jeremy Fitzhardinge <jeremy@goop.org> wrote: >>>> >>>> >>>> >>>>> On 06/10/2010 03:13 AM, Mukesh Rathor wrote: >>>>> >>>>> >>>>>> Well, BUG_ON is only triggered if booting more than 32 VCPUs on >>>>>> a *very old* xen (pre xen 3.1.0). >>>>>> >>>>>> Looking at code closely, we could just set setup_max_cpus to 32 >>>>>> some where in xen function, perhaps even in xen_vcpu_setup(). >>>>>> That way later in smp_init() it would just be ok. >>>>>> >>>>>> >>>>>> >>>>> Yes. >>>>> >>>>> >>>>> >>>>>> One thing tho, the per cpus areas are already setup at that >>>>>> point, so that would need to be cleaned. BTW, I don''t >>>>>> understand why have_vcpu_info_placement is set to 0 in >>>>>> xen_guest_init()? >>>>>> >>>>>> >>>>> xen_guest_init is used by the pvhvm path, and hvm domains don''t >>>>> have a notion of vcpu info placement. >>>>> >>>>> >>>>> >>>>>> What minimum version of xen is required to run pvops kernel? >>>>>> >>>>>> >>>>>> >>>>> In theory it should be back-compatible for all Xen 3, but in >>>>> practice it tweaks lots of bugs in older Xens (particularly >>>>> 32-on-64). I don''t know that anyone has definitively established >>>>> an earliest version. I implemented vcpu info placement for use >>>>> in pvops kernels, but it was never my intention that it be an >>>>> absolute requirement. >>>>> >>>>> J >>>>> >>>>> >>>> Ok, attached patch without BUG_ON. Please feel free to modify >>>> to your liking also. >>>> >>>> >>> It looks like you smashed all the tabs into spaces so its hard to >>> see what you''ve changed in the diff. I''ll fix it up and give it a >>> look-over. >>> >>> J >>> >> Sorry, I''ve tabs turned off because patches I submit to other product >> I work on must be tab free. Anyways, re attached a new one with tabs. >> >> thanks again, >> Mukesh >> >> Signed-off-by: Mukesh Rathor <mukesh.rathor@oracle.com> >> > Hi Jeremy, > > Just curious, did this patch ever make it? >Probably not. Looks like I forgot to tag it as "mail containing patch" so it fell through the cracks. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Mukesh Rathor
2010-Jul-17 01:11 UTC
[Xen-devel] Re: [PATCHEs]: support more than 32 VCPUs in guests
On Fri, 16 Jul 2010 18:09:44 -0700 Jeremy Fitzhardinge <jeremy@goop.org> wrote:> On 07/16/2010 06:06 PM, Mukesh Rathor wrote: > > On Tue, 15 Jun 2010 11:45:43 -0700 > > Mukesh Rathor <mukesh.rathor@oracle.com> wrote: > > > > > >> On Tue, 15 Jun 2010 09:30:35 +0100 > >> Jeremy Fitzhardinge <jeremy@goop.org> wrote: > >> > >> > >>> On 06/15/2010 03:49 AM, Mukesh Rathor wrote: > >>> > >>>> On Mon, 14 Jun 2010 10:37:30 +0100 > >>>> Jeremy Fitzhardinge <jeremy@goop.org> wrote: > >>>> > >>>> > >>>> > >>>>> On 06/10/2010 03:13 AM, Mukesh Rathor wrote: > >>>>> > >>>>> > >>>>>> Well, BUG_ON is only triggered if booting more than 32 VCPUs on > >>>>>> a *very old* xen (pre xen 3.1.0). > >>>>>> > >>>>>> Looking at code closely, we could just set setup_max_cpus to 32 > >>>>>> some where in xen function, perhaps even in xen_vcpu_setup(). > >>>>>> That way later in smp_init() it would just be ok. > >>>>>> > >>>>>> > >>>>>> > >>>>> Yes. > >>>>> > >>>>> > >>>>> > >>>>>> One thing tho, the per cpus areas are already setup at that > >>>>>> point, so that would need to be cleaned. BTW, I don''t > >>>>>> understand why have_vcpu_info_placement is set to 0 in > >>>>>> xen_guest_init()? > >>>>>> > >>>>>> > >>>>> xen_guest_init is used by the pvhvm path, and hvm domains don''t > >>>>> have a notion of vcpu info placement. > >>>>> > >>>>> > >>>>> > >>>>>> What minimum version of xen is required to run pvops kernel? > >>>>>> > >>>>>> > >>>>>> > >>>>> In theory it should be back-compatible for all Xen 3, but in > >>>>> practice it tweaks lots of bugs in older Xens (particularly > >>>>> 32-on-64). I don''t know that anyone has definitively > >>>>> established an earliest version. I implemented vcpu info > >>>>> placement for use in pvops kernels, but it was never my > >>>>> intention that it be an absolute requirement. > >>>>> > >>>>> J > >>>>> > >>>>> > >>>> Ok, attached patch without BUG_ON. Please feel free to modify > >>>> to your liking also. > >>>> > >>>> > >>> It looks like you smashed all the tabs into spaces so its hard to > >>> see what you''ve changed in the diff. I''ll fix it up and give it a > >>> look-over. > >>> > >>> J > >>> > >> Sorry, I''ve tabs turned off because patches I submit to other > >> product I work on must be tab free. Anyways, re attached a new > >> one with tabs. > >> > >> thanks again, > >> Mukesh > >> > >> Signed-off-by: Mukesh Rathor <mukesh.rathor@oracle.com> > >> > > Hi Jeremy, > > > > Just curious, did this patch ever make it? > > > > Probably not. Looks like I forgot to tag it as "mail containing > patch" so it fell through the cracks. > > JDid you want me to resubmit it, or can you find the patch? thanks, m _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-Jul-26 22:57 UTC
[Xen-devel] Re: [PATCHEs]: support more than 32 VCPUs in guests
On 06/15/2010 11:45 AM, Mukesh Rathor wrote:> On Tue, 15 Jun 2010 09:30:35 +0100 > Jeremy Fitzhardinge<jeremy@goop.org> wrote: > >> On 06/15/2010 03:49 AM, Mukesh Rathor wrote: >>> On Mon, 14 Jun 2010 10:37:30 +0100 >>> Jeremy Fitzhardinge<jeremy@goop.org> wrote: >>> >>> >>>> On 06/10/2010 03:13 AM, Mukesh Rathor wrote: >>>> >>>>> Well, BUG_ON is only triggered if booting more than 32 VCPUs on a >>>>> *very old* xen (pre xen 3.1.0). >>>>> >>>>> Looking at code closely, we could just set setup_max_cpus to 32 >>>>> some where in xen function, perhaps even in xen_vcpu_setup(). >>>>> That way later in smp_init() it would just be ok. >>>>> >>>>> >>>> Yes. >>>> >>>> >>>>> One thing tho, the per cpus areas are already setup at that point, >>>>> so that would need to be cleaned. BTW, I don''t understand why >>>>> have_vcpu_info_placement is set to 0 in xen_guest_init()? >>>>> >>>>> >>>> xen_guest_init is used by the pvhvm path, and hvm domains don''t >>>> have a notion of vcpu info placement. >>>> >>>> >>>>> What minimum version of xen is required to run pvops kernel? >>>>> >>>>> >>>> In theory it should be back-compatible for all Xen 3, but in >>>> practice it tweaks lots of bugs in older Xens (particularly >>>> 32-on-64). I don''t know that anyone has definitively established >>>> an earliest version. I implemented vcpu info placement for use in >>>> pvops kernels, but it was never my intention that it be an >>>> absolute requirement. >>>> >>>> J >>>> >>> Ok, attached patch without BUG_ON. Please feel free to modify >>> to your liking also. >>> >> It looks like you smashed all the tabs into spaces so its hard to see >> what you''ve changed in the diff. I''ll fix it up and give it a >> look-over. >> >> J > Sorry, I''ve tabs turned off because patches I submit to other product I > work on must be tab free. Anyways, re attached a new one with tabs.This doesn''t compile with CONFIG_SMP=n because setup_max_cpus doesn''t exist. For now I''ve just put a couple of #ifdef CONFIG_SMPs in there to avoid compile errors, but could you look at coming up with a cleaner solution? Thanks, J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-Jul-27 00:37 UTC
Re: [Xen-devel] Re: [PATCHEs]: support more than 32 VCPUs in guests
On 07/26/2010 03:57 PM, Jeremy Fitzhardinge wrote:> On 06/15/2010 11:45 AM, Mukesh Rathor wrote: >> On Tue, 15 Jun 2010 09:30:35 +0100 >> Jeremy Fitzhardinge<jeremy@goop.org> wrote: >> >>> On 06/15/2010 03:49 AM, Mukesh Rathor wrote: >>>> On Mon, 14 Jun 2010 10:37:30 +0100 >>>> Jeremy Fitzhardinge<jeremy@goop.org> wrote: >>>> >>>> >>>>> On 06/10/2010 03:13 AM, Mukesh Rathor wrote: >>>>> >>>>>> Well, BUG_ON is only triggered if booting more than 32 VCPUs on a >>>>>> *very old* xen (pre xen 3.1.0). >>>>>> >>>>>> Looking at code closely, we could just set setup_max_cpus to 32 >>>>>> some where in xen function, perhaps even in xen_vcpu_setup(). >>>>>> That way later in smp_init() it would just be ok. >>>>>> >>>>>> >>>>> Yes. >>>>> >>>>> >>>>>> One thing tho, the per cpus areas are already setup at that point, >>>>>> so that would need to be cleaned. BTW, I don''t understand why >>>>>> have_vcpu_info_placement is set to 0 in xen_guest_init()? >>>>>> >>>>>> >>>>> xen_guest_init is used by the pvhvm path, and hvm domains don''t >>>>> have a notion of vcpu info placement. >>>>> >>>>> >>>>>> What minimum version of xen is required to run pvops kernel? >>>>>> >>>>>> >>>>> In theory it should be back-compatible for all Xen 3, but in >>>>> practice it tweaks lots of bugs in older Xens (particularly >>>>> 32-on-64). I don''t know that anyone has definitively established >>>>> an earliest version. I implemented vcpu info placement for use in >>>>> pvops kernels, but it was never my intention that it be an >>>>> absolute requirement. >>>>> >>>>> J >>>>> >>>> Ok, attached patch without BUG_ON. Please feel free to modify >>>> to your liking also. >>>> >>> It looks like you smashed all the tabs into spaces so its hard to see >>> what you''ve changed in the diff. I''ll fix it up and give it a >>> look-over. >>> >>> J >> Sorry, I''ve tabs turned off because patches I submit to other product I >> work on must be tab free. Anyways, re attached a new one with tabs. > > This doesn''t compile with CONFIG_SMP=n because setup_max_cpus doesn''t > exist. For now I''ve just put a couple of #ifdef CONFIG_SMPs in there > to avoid compile errors, but could you look at coming up with a > cleaner solution?Hm, I committed a tidier version. I don''t see it getting much better. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel