On modern mainline, when a PV on HVM kernel boots, it will detect the
ability to PCI unplug the PCI SD devices, so to avoid producing
/dev/xvdX and /dev/sdX referring to the same device. On Xen Dom0 prior
to 3.4.something, this fails because PCI unplug is not supported.
The check can be worked around by using emulunplug=unnecessary, but
this is not optimal as
a) you still end up with 2 devices; and
b) it requires modifying the kernel boot line, which is not always
easy (e.g. if your bootloader is supplied by a third party IAAS
provider and you wish to use a mainline kernel).
Another approach would be to stop sd_mod loading another way
(e.g. blacklist sd_mod). This does not work well as many kernels have
sd_mod built in.
I am trying to think of a non-intrusive way to prevent sd_mod initialising
if we are doing PV on HVM and PCI unplug is not supported (other than
upgrade Xen). One possibility would be to manually register dummy
block major device with the same major number as used by sd. This would
cause register_blkdev to return EBUSY and the module not to init. However,
it would leave the dummy block device around. Another more intrusive
method would simply be to put a check into sd_mod. Yet more intrusive
would be to modify the entire do_initcalls/module_init subsystem
to provide a generalised method for one bit of code to disable another.
Has someone already looked into this and concluded it''s impossible? Or
is this worth a little time?
--
Alex Bligh
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Paolo Bonzini
2011-Jan-12 13:04 UTC
[Xen-devel] Re: 2.6.37 PV on HVM and emul_unplug=unnecessary
On 01/12/2011 12:45 PM, Alex Bligh wrote:> On modern mainline, when a PV on HVM kernel boots, it will detect the > ability to PCI unplug the PCI SD devices, so to avoid producing > /dev/xvdX and /dev/sdX referring to the same device. On Xen Dom0 prior > to 3.4.something, this fails because PCI unplug is not supported. > The check can be worked around by using emulunplug=unnecessary, but > this is not optimal as > a) you still end up with 2 devices; and > b) it requires modifying the kernel boot line, which is not always > easy (e.g. if your bootloader is supplied by a third party IAAS > provider and you wish to use a mainline kernel). > > Another approach would be to stop sd_mod loading another way > (e.g. blacklist sd_mod). This does not work well as many kernels have > sd_mod built in. > > I am trying to think of a non-intrusive way to prevent sd_mod initialising > if we are doing PV on HVM and PCI unplug is not supported (other than > upgrade Xen). One possibility would be to manually register dummy > block major device with the same major number as used by sd. This would > cause register_blkdev to return EBUSY and the module not to init. However, > it would leave the dummy block device around.On Windows it is possible to add a "filter driver" above the PCI and IDE buses and hide the emulated devices this way. It doesn''t leave a dummy disk, but it does leave a dummy device in the PCI tree. It''s a kind of kernel-level blacklist. Also, it couldn''t be an external module if sd_mod is built in, which makes it much less interesting.> Has someone already looked into this and concluded it''s impossible? Or > is this worth a little time?Something similar was done in the 2.6.18 "XenoLinux" kernels, but it''s ugly and it touches into areas of the kernell that Xen support shouldn''t affect. If your "Xen Dom0 prior to 3.4.something" is RHEL/CentOS, the 5.6 version will include a backport of PCI unplug support. Then you only need to bug your cloud provider. :) Paolo _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Alex Bligh
2011-Jan-12 13:25 UTC
[Xen-devel] Re: 2.6.37 PV on HVM and emul_unplug=unnecessary
--On 12 January 2011 14:04:57 +0100 Paolo Bonzini <pbonzini@redhat.com> wrote:> Something similar was done in the 2.6.18 "XenoLinux" kernels, but it''s > ugly and it touches into areas of the kernell that Xen support shouldn''t > affect. > > If your "Xen Dom0 prior to 3.4.something" is RHEL/CentOS, the 5.6 version > will include a backport of PCI unplug support. Then you only need to bug > your cloud provider. :)I am the cloud provider. The real fix is to move to Xen 4.0.1. Part of the issue, however, is moving to 4.0.1 will make a subset of guests behave differently from others, as suddenly their /dev/sda will disappear. We know there are people who don''t use standard installs so possibly unknowingly are not using pv drivers (e.g. they have /dev/sdaX in their fstab). It would be nice if I could separate this change from the rest of the 4.0.1 upgrade, as this would make a 4.0.1 upgrade less scary. It would be oh-so-much-easier-for-me if /dev/sdaX disappeared when they upgraded their kernel, rather than when I make a later change. However, if could point me at a working changeset which contains the backport, that would be really useful. Stefano posted one which I think he marked as "untested". -- Alex Bligh _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel