Xen devel - Aug 2010 - shutdown/reboot/sysrq/vcpu-set from dom0 to HVM guest

Hello,

I''m working with HVM linux guest and found that "xl
reboot/shutdown/sysrq/vcpu-set DOMAIN" did not work.

Xen : 4.0.1
Dom0 : 2.6.32.
DomU : 2.6.36-rc3    pv-on-hvm drivers (vbd, vif, xenbus)

In order to make the commands working in the file
linux-2.6.36-rc3/drivers/xen/manage.c  I''ve modified the function
__setup_shutdown_even(void) to remove the check (this is BAD !).


For the cpu-set I''ve modified the file
linux-2.6.36-rc3/drivers/xen/cpu_hotplug.c and the check fo pv_domain in
setup_vcpu_hotlug_event(void).  And then I can remove CPUs but not add some  :

    xl vcpu-set DOMID 2 -> "CPU 2 is now offline / CPU 3 is now
offline"

I''ve tried the same modification on ballooning driver and hvc console
one but it gave me a kernel panic ...



Regards,

Frederic. 
 
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, 31 Aug 2010, Fr??d??ric VANNI??RE wrote:
> Hello,
> 
> I''m working with HVM linux guest and found that "xl
reboot/shutdown/sysrq/vcpu-set DOMAIN" did not work.
> 
> Xen : 4.0.1
> Dom0 : 2.6.32.
> DomU : 2.6.36-rc3    pv-on-hvm drivers (vbd, vif, xenbus)
> 
> In order to make the commands working in the file
linux-2.6.36-rc3/drivers/xen/manage.c  I''ve modified the function
__setup_shutdown_even(void) to remove the check (this is BAD !).
> 
this is due to a difference between Jeremy''s 2.6.32 tree and my pv on
hvm tree: xen_setup_shutdown_event calls register_xenstore_notifier.
In my tree register_xenstore_notifier calls the notifier function
immediately if xenstore is ready while on Jeremy''s tree it does not.
Considering that xen_setup_shutdown_event is called from 
drivers/xen/platform-pci.c after xenbus_probe, the notifier function
will never get called.

Moving xen_setup_shutdown_event one line up, above xenbus_probe in
drivers/xen/platform-pci.c, should fix the issue but it is not the proper
fix for it.

> 
> For the cpu-set I''ve modified the file
linux-2.6.36-rc3/drivers/xen/cpu_hotplug.c and the check fo pv_domain in
setup_vcpu_hotlug_event(void).  And then I can remove CPUs but not add some  :
> 
>     xl vcpu-set DOMID 2 -> "CPU 2 is now offline / CPU 3 is now
offline"
> 
> I''ve tried the same modification on ballooning driver and hvc
console one but it gave me a kernel panic ...
> 
In my tree both the ballooning driver and the hvc console should work
correctly on hvm.
Regarding the pv cpu hotplug mechanism, I don''t think is a good idea to
enable it on hvm: there must be an alternative fully virtualized way to
hotplug a cpu into a non-pv guest, like Windows, and that should work
for pv on hvm guests too.

BTW the pv on hvm tree I am referring to it available here:

git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git

branch name 2.6.35-rc5-pvhvm-v7.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, 1 Sep 2010, Stefano Stabellini wrote:
> > 
> > For the cpu-set I''ve modified the file
linux-2.6.36-rc3/drivers/xen/cpu_hotplug.c and the check fo pv_domain in
setup_vcpu_hotlug_event(void).  And then I can remove CPUs but not add some  :
> > 
> >     xl vcpu-set DOMID 2 -> "CPU 2 is now offline / CPU 3 is
now offline"
> > 
> > I''ve tried the same modification on ballooning driver and hvc
console one but it gave me a kernel panic ...
> > 
> 
> In my tree both the ballooning driver and the hvc console should work
> correctly on hvm.
> Regarding the pv cpu hotplug mechanism, I don''t think is a good
idea to
> enable it on hvm: there must be an alternative fully virtualized way to
> hotplug a cpu into a non-pv guest, like Windows, and that should work
> for pv on hvm guests too.
 
I have done some testing on cpu hotplug for hvm guests and it turns out
that it is currently broken on xen-unstable.
I have just sent two patches to fix it.
Once they got applied you should be able to specify the maximum number
of vcpus using the "maxvcpus" parameter in your config file; then use
vcpu-set to dynamically change the number of vcpus in the guest.
Linux needs to be compiled with CONFIG_ACPI_HOTPLUG_CPU=y in order for
this to work.
Once you hotplug a new cpu in the guest you also need to enable it
manually, with the following command:

echo -n 1 > /sys/devices/system/cpu/cpu$CPUNUMBER/online


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, 2010-09-02 at 14:21 +0100, Stefano Stabellini
wrote:
> On Wed, 1 Sep 2010, Stefano Stabellini wrote:
> > > 
> > > For the cpu-set I''ve modified the file
linux-2.6.36-rc3/drivers/xen/cpu_hotplug.c and the check fo pv_domain in
setup_vcpu_hotlug_event(void).  And then I can remove CPUs but not add some  :
> > > 
> > >     xl vcpu-set DOMID 2 -> "CPU 2 is now offline / CPU 3
is now offline"
> > > 
> > > I''ve tried the same modification on ballooning driver
and hvc console one but it gave me a kernel panic ...
> > > 
> > 
> > In my tree both the ballooning driver and the hvc console should work
> > correctly on hvm.
> > Regarding the pv cpu hotplug mechanism, I don''t think is a
good idea to
> > enable it on hvm: there must be an alternative fully virtualized way
to
> > hotplug a cpu into a non-pv guest, like Windows, and that should work
> > for pv on hvm guests too.
>  
> I have done some testing on cpu hotplug for hvm guests and it turns out
> that it is currently broken on xen-unstable.
> I have just sent two patches to fix it.
> Once they got applied you should be able to specify the maximum number
> of vcpus using the "maxvcpus" parameter in your config file; then
use
> vcpu-set to dynamically change the number of vcpus in the guest.
> Linux needs to be compiled with CONFIG_ACPI_HOTPLUG_CPU=y in order for
> this to work.
> Once you hotplug a new cpu in the guest you also need to enable it
> manually, with the following command:
> 
> echo -n 1 > /sys/devices/system/cpu/cpu$CPUNUMBER/online
FWIW putting this in /etc/udev/rules.d/ somewhere can automate this for
you:

        ACTION=="add", SUBSYSTEM=="cpu", RUN+="/bin/sh
-c ''[ ! -e /sys$devpath/online ] || echo 1 >
/sys$devpath/online''"

Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Stefano Stabellini wrote:
> On Wed, 1 Sep 2010, Stefano Stabellini wrote:
>>> 
>>> For the cpu-set I''ve modified the file
>>> linux-2.6.36-rc3/drivers/xen/cpu_hotplug.c and the check fo
>>> pv_domain in setup_vcpu_hotlug_event(void).  And then I can remove
>>> CPUs but not add some  :   
>>> 
>>>     xl vcpu-set DOMID 2 -> "CPU 2 is now offline / CPU 3 is
now
>>> offline" 
>>> 
>>> I''ve tried the same modification on ballooning driver and
hvc
>>> console one but it gave me a kernel panic ... 
>>> 
>> 
>> In my tree both the ballooning driver and the hvc console should
>> work correctly on hvm. Regarding the pv cpu hotplug mechanism, I
>> don''t think is a good idea to enable it on hvm: there must be
an
>> alternative fully virtualized way to hotplug a cpu into a non-pv
>> guest, like Windows, and that should work for pv on hvm guests too.
> 
> I have done some testing on cpu hotplug for hvm guests and it turns
> out that it is currently broken on xen-unstable.
Some strange.
Stabellini, would you please tell me more clear?
How did you do the test? what command? and error log?

Thanks,
Jinsong
> I have just sent two patches to fix it.
> Once they got applied you should be able to specify the maximum number
> of vcpus using the "maxvcpus" parameter in your config file; then
use
> vcpu-set to dynamically change the number of vcpus in the guest.
> Linux needs to be compiled with CONFIG_ACPI_HOTPLUG_CPU=y in order for
> this to work.
> Once you hotplug a new cpu in the guest you also need to enable it
> manually, with the following command:
> 
> echo -n 1 > /sys/devices/system/cpu/cpu$CPUNUMBER/online
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, 3 Sep 2010, Liu, Jinsong wrote:
> Some strange.
> Stabellini, would you please tell me more clear?
> How did you do the test? what command? and error log?
> 
Qemu doesn''t receive the xenstore watch events properly.
I think it might have something to do with the way xl implements
vcpu-set, that is probably subtly different from the way xend does it.
Anyway, the patch "qemu-xen: fix cpu hotplug" fixes the problem.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Stefano Stabellini wrote:
> On Fri, 3 Sep 2010, Liu, Jinsong wrote:
>> Some strange.
>> Stabellini, would you please tell me more clear?
>> How did you do the test? what command? and error log?
>> 
> 
> Qemu doesn''t receive the xenstore watch events properly.
> I think it might have something to do with the way xl implements
Yes, it seems a little bug in xl path :)
> vcpu-set, that is probably subtly different from the way xend does it.
> Anyway, the patch "qemu-xen: fix cpu hotplug" fixes the problem.
Yes, the patch is OK, but I think it no need to do so.
How about still keep old way, to watch ''common'' cpu node?

Jinsong
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, 3 Sep 2010, Liu, Jinsong wrote:
> Yes, the patch is OK, but I think it no need to do so.
> How about still keep old way, to watch ''common'' cpu node?
> 
Yeah, you are right let''s keep it as it is.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel