Jan Beulich
2010-May-11 09:34 UTC
[Xen-devel] cpu_down() but no cpu_up() in drivers/xen/cpu_hotplug.c ?
Jeremy, how is pv-ops bringing up CPUs set to online in xenstore? Is this not an automatic process (i.e. requires writing to respective online node in sysfs), like in the traditional tree? Thanks, Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2010-May-11 09:45 UTC
Re: [Xen-devel] cpu_down() but no cpu_up() in drivers/xen/cpu_hotplug.c ?
On Tue, 2010-05-11 at 10:34 +0100, Jan Beulich wrote:> Jeremy, > > how is pv-ops bringing up CPUs set to online in xenstore? Is this not > an automatic process (i.e. requires writing to respective online node > in sysfs), like in the traditional tree?The original commit which added CPU hotplug to pvops says: xen: implement CPU hotplugging Note the changes from 2.6.18-xen CPU hotplugging: A vcpu_down request from the remote admin via Xenbus both hotunplugs the CPU, and disables it by removing it from the cpu_present map, and removing its entry in /sys. A vcpu_up request from the remote admin only re-enables the CPU, and does not immediately bring the CPU up. A udev event is emitted, which can be caught by the user if he wishes to automatically re-up CPUs when available, or implement a more complex policy. Signed-off-by: Alex Nixon <alex.nixon@citrix.com> Acked-by: Jeremy Fitzhardinge <jeremy@goop.org> Signed-off-by: Ingo Molnar <mingo@elte.hu> I''m not sure how the decision was reached to implement it this way, perhaps for consistency with CPU hotplug on other platforms/architectures? FWIW I use a udev rule to bring up CPUs as they are added, which is equivalent to the old behaviour: ACTION=="add", SUBSYSTEM=="cpu", RUN+="/bin/sh -c ''[ ! -e /sys$devpath/online ] || echo 1 > /sys$devpath/online''" Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Dietmar Hahn
2010-May-11 11:30 UTC
Re: [Xen-devel] cpu_down() but no cpu_up() in drivers/xen/cpu_hotplug.c ?
Am 11.05.2010 schrieb Ian Campbell:> On Tue, 2010-05-11 at 10:34 +0100, Jan Beulich wrote: > > Jeremy, > > > > how is pv-ops bringing up CPUs set to online in xenstore? Is this not > > an automatic process (i.e. requires writing to respective online node > > in sysfs), like in the traditional tree? > > The original commit which added CPU hotplug to pvops says: > > xen: implement CPU hotplugging > > Note the changes from 2.6.18-xen CPU hotplugging: > > A vcpu_down request from the remote admin via Xenbus both hotunplugs the > CPU, and disables it by removing it from the cpu_present map, and removing > its entry in /sys. > > A vcpu_up request from the remote admin only re-enables the CPU, and does > not immediately bring the CPU up. A udev event is emitted, which can be > caught by the user if he wishes to automatically re-up CPUs when available, > or implement a more complex policy. > > Signed-off-by: Alex Nixon <alex.nixon@citrix.com> > Acked-by: Jeremy Fitzhardinge <jeremy@goop.org> > Signed-off-by: Ingo Molnar <mingo@elte.hu> > > I''m not sure how the decision was reached to implement it this way, > perhaps for consistency with CPU hotplug on other > platforms/architectures? > > FWIW I use a udev rule to bring up CPUs as they are added, which is > equivalent to the old behaviour: > > ACTION=="add", SUBSYSTEM=="cpu", RUN+="/bin/sh -c ''[ ! -e /sys$devpath/online ] || echo 1 > /sys$devpath/online''" > > Ian.Maybe it would be good then to have a comment somewhere in the tree with this udev rule as a hint? Thanks. Dietmar. -- Company details: http://ts.fujitsu.com/imprint.html _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-May-11 17:40 UTC
Re: [Xen-devel] cpu_down() but no cpu_up() in drivers/xen/cpu_hotplug.c ?
On 05/11/2010 02:45 AM, Ian Campbell wrote:> The original commit which added CPU hotplug to pvops says: > > xen: implement CPU hotplugging > > Note the changes from 2.6.18-xen CPU hotplugging: > > A vcpu_down request from the remote admin via Xenbus both hotunplugs the > CPU, and disables it by removing it from the cpu_present map, and removing > its entry in /sys. > > A vcpu_up request from the remote admin only re-enables the CPU, and does > not immediately bring the CPU up. A udev event is emitted, which can be > caught by the user if he wishes to automatically re-up CPUs when available, > or implement a more complex policy. > > Signed-off-by: Alex Nixon <alex.nixon@citrix.com> > Acked-by: Jeremy Fitzhardinge <jeremy@goop.org> > Signed-off-by: Ingo Molnar <mingo@elte.hu> > > I''m not sure how the decision was reached to implement it this way, > perhaps for consistency with CPU hotplug on other > platforms/architectures? >Yes, it was to make it consistent with native physical CPU hotplug. It also replaced some other xen-specific mechanism to allow the domain to control when the cpu was actually added (I forget the details; something like "cpus allowed" vs "cpus active" or something?).> FWIW I use a udev rule to bring up CPUs as they are added, which is > equivalent to the old behaviour: > > ACTION=="add", SUBSYSTEM=="cpu", RUN+="/bin/sh -c ''[ ! -e /sys$devpath/online ] || echo 1 > /sys$devpath/online''" >Fedora and RHEL have been shipping with something like this for a while. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jiang, Yunhong
2010-May-12 03:25 UTC
RE: [Xen-devel] cpu_down() but no cpu_up() in drivers/xen/cpu_hotplug.c ?
>-----Original Message----- >From: xen-devel-bounces@lists.xensource.com >[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Jeremy Fitzhardinge >Sent: Wednesday, May 12, 2010 1:40 AM >To: Ian Campbell >Cc: xen-devel@lists.xensource.com; Jan Beulich >Subject: Re: [Xen-devel] cpu_down() but no cpu_up() in drivers/xen/cpu_hotplug.c ? > >On 05/11/2010 02:45 AM, Ian Campbell wrote: >> The original commit which added CPU hotplug to pvops says: >> >> xen: implement CPU hotplugging >> >> Note the changes from 2.6.18-xen CPU hotplugging: >> >> A vcpu_down request from the remote admin via Xenbus both hotunplugs the >> CPU, and disables it by removing it from the cpu_present map, and removing >> its entry in /sys. >> >> A vcpu_up request from the remote admin only re-enables the CPU, and does >> not immediately bring the CPU up. A udev event is emitted, which can be >> caught by the user if he wishes to automatically re-up CPUs when available, >> or implement a more complex policy. >> >> Signed-off-by: Alex Nixon <alex.nixon@citrix.com> >> Acked-by: Jeremy Fitzhardinge <jeremy@goop.org> >> Signed-off-by: Ingo Molnar <mingo@elte.hu> >> >> I'm not sure how the decision was reached to implement it this way, >> perhaps for consistency with CPU hotplug on other >> platforms/architectures? >> > >Yes, it was to make it consistent with native physical CPU hotplug. It >also replaced some other xen-specific mechanism to allow the domain to >control when the cpu was actually added (I forget the details; something >like "cpus allowed" vs "cpus active" or something?).I remember for cpu remove, the xen's vcpu is different to native method. In native, it will only trigger a uevent to user space (at least in version like 2.6.31), while for xen vcpu, it will remove the vcpu directly. --jyh> >> FWIW I use a udev rule to bring up CPUs as they are added, which is >> equivalent to the old behaviour: >> >> ACTION=="add", SUBSYSTEM=="cpu", RUN+="/bin/sh -c '[ ! -e >/sys$devpath/online ] || echo 1 > /sys$devpath/online'" >> > >Fedora and RHEL have been shipping with something like this for a while. > > J > >_______________________________________________ >Xen-devel mailing list >Xen-devel@lists.xensource.com >http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-May-12 16:26 UTC
Re: [Xen-devel] cpu_down() but no cpu_up() in drivers/xen/cpu_hotplug.c ?
On 05/11/2010 08:25 PM, Jiang, Yunhong wrote:>> Yes, it was to make it consistent with native physical CPU hotplug. It >> also replaced some other xen-specific mechanism to allow the domain to >> control when the cpu was actually added (I forget the details; something >> like "cpus allowed" vs "cpus active" or something?). >> > I remember for cpu remove, the xen''s vcpu is different to native method. In native, it will only trigger a uevent to user space (at least in version like 2.6.31), while for xen vcpu, it will remove the vcpu directly. >I would think that Xen and native are much the same; if you pull out a physical CPU from the system, that''s in no way advisory ;) Similarly, if you remove a vcpu from a guest, that''s an external policy being imposed onto the guest, and it doesn''t get much say in the matter, beyond being able to clean up before the vcpu goes away. If the domain wants to stop using a vcpu, it can simply do that by soft-downing the vcpu, but it remains attached to the domain (again, analogous to the native case). J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jiang, Yunhong
2010-May-13 02:48 UTC
RE: [Xen-devel] cpu_down() but no cpu_up() in drivers/xen/cpu_hotplug.c ?
>-----Original Message----- >From: Jeremy Fitzhardinge [mailto:jeremy@goop.org] >Sent: Thursday, May 13, 2010 12:27 AM >To: Jiang, Yunhong >Cc: Ian Campbell; xen-devel@lists.xensource.com; Jan Beulich >Subject: Re: [Xen-devel] cpu_down() but no cpu_up() in drivers/xen/cpu_hotplug.c ? > >On 05/11/2010 08:25 PM, Jiang, Yunhong wrote: >>> Yes, it was to make it consistent with native physical CPU hotplug. It >>> also replaced some other xen-specific mechanism to allow the domain to >>> control when the cpu was actually added (I forget the details; something >>> like "cpus allowed" vs "cpus active" or something?). >>> >> I remember for cpu remove, the xen's vcpu is different to native method. In native, >it will only trigger a uevent to user space (at least in version like 2.6.31), while for >xen vcpu, it will remove the vcpu directly. >> > >I would think that Xen and native are much the same; if you pull out aI suspect if native has tested the CPU hotmove. But at least from the code in container_notify_cb() at drivers/acpi/container.c, seems it will trigger a uevent to user space and the user space will take action (mostly through eiject entry under sysfs, I think). But still I don't know if this has been tested on native environment. So maybe the difference from xen vcpu does not matter. case ACPI_NOTIFY_DEVICE_CHECK: printk(KERN_WARNING "Container driver received %s event\n", (type == ACPI_NOTIFY_BUS_CHECK) ? "ACPI_NOTIFY_BUS_CHECK" : "ACPI_NOTIFY_DEVICE_CHECK"); status = acpi_bus_get_device(handle, &device); if (present) { if (ACPI_FAILURE(status) || !device) { result = container_device_add(&device, handle); if (!result) kobject_uevent(&device->dev.kobj, KOBJ_ONLINE); else printk(KERN_WARNING "Failed to add container\n"); } } else { if (ACPI_SUCCESS(status)) { /* device exist and this is a remove request */ kobject_uevent(&device->dev.kobj, KOBJ_OFFLINE); } }>physical CPU from the system, that's in no way advisory ;) Similarly, >if you remove a vcpu from a guest, that's an external policy being >imposed onto the guest, and it doesn't get much say in the matter, >beyond being able to clean up before the vcpu goes away. > >If the domain wants to stop using a vcpu, it can simply do that by >soft-downing the vcpu, but it remains attached to the domain (again, >analogous to the native case). > > J_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel