Pasi Kärkkäinen
2010-Mar-27 17:32 UTC
[Xen-devel] Automatic loading of xen-evtchn module in Xen 4.0.0?
Hello,
xend automatically mounts /proc/xen, but it doesn''t try loading evtchn
or xen-evtchn modules.
Should we add automatic loading of these modules?
xend doesn''t start if xen event channel functionality is missing.
Fedora Xen rpms do load modules automatically, like this:
# cat /etc/sysconfig/modules/xen.modules
#!/bin/sh
test -d /proc/xen || exit 0
test -f /proc/xen/capabilities || mount -t xenfs xen /proc/xen
test -f /proc/xen/capabilities || exit 0
grep -q "control_d" /proc/xen/capabilities || exit 0
for module in xen-evtchn evtchn blkbk netbk; do
modprobe ${module} >/dev/null 2>&1
done
-- Pasi
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-Mar-27 22:18 UTC
Re: [Xen-devel] Automatic loading of xen-evtchn module in Xen 4.0.0?
On 03/27/2010 10:32 AM, Pasi Kärkkäinen wrote:> Hello, > > xend automatically mounts /proc/xen, but it doesn''t try loading evtchn or xen-evtchn modules. > Should we add automatic loading of these modules? > > xend doesn''t start if xen event channel functionality is missing. > > Fedora Xen rpms do load modules automatically, like this: > > # cat /etc/sysconfig/modules/xen.modules > #!/bin/sh > > test -d /proc/xen || exit 0 > test -f /proc/xen/capabilities || mount -t xenfs xen /proc/xen > test -f /proc/xen/capabilities || exit 0 > grep -q "control_d" /proc/xen/capabilities || exit 0 > > for module in xen-evtchn evtchn blkbk netbk; do > modprobe ${module}>/dev/null 2>&1 > done >We should be able to make these mount automatically with udev rules, I think, but I''m not sure whether you can do udev rules to mount misc devices (which is what /dev/xen/evtchn is). I think the front/back xenbus and device drivers are still a bit of a mess with regard to modules; I have to admit I tend to just compile them all in. It would be nice to get all that properly sorted out so they can be sensibly modularized and auto-loaded as required. (A good first step would be to systematically work out what compile combinations actually work or fail, hint, hint ;) J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2010-Mar-30 09:44 UTC
Re: [Xen-devel] Automatic loading of xen-evtchn module in Xen 4.0.0?
On Sat, 2010-03-27 at 22:18 +0000, Jeremy Fitzhardinge wrote:> On 03/27/2010 10:32 AM, Pasi Kärkkäinen wrote: > > Hello, > > > > xend automatically mounts /proc/xen, but it doesn''t try loading evtchn or xen-evtchn modules. > > Should we add automatic loading of these modules? > > > > xend doesn''t start if xen event channel functionality is missing. > > > > Fedora Xen rpms do load modules automatically, like this: > > > > # cat /etc/sysconfig/modules/xen.modules > > #!/bin/sh > > > > test -d /proc/xen || exit 0 > > test -f /proc/xen/capabilities || mount -t xenfs xen /proc/xen > > test -f /proc/xen/capabilities || exit 0 > > grep -q "control_d" /proc/xen/capabilities || exit 0 > > > > for module in xen-evtchn evtchn blkbk netbk; do > > modprobe ${module}>/dev/null 2>&1 > > done > > > > We should be able to make these mount automatically with udev rules, I > think, but I''m not sure whether you can do udev rules to mount misc > devices (which is what /dev/xen/evtchn is).I think you mean "modprobe misc devices" not mount. I see various uses of the MODULE_ALIAS_MISCDEV macro in the kernel -- but I''m not sure what actually causes a matching hotplug event to be generated, other than trying to open a matching device node which with udev won''t exist until the module is loaded. Perhaps something in the xen core code (or xenbus?) could generate the necessary hotplug events for the Xen "platform devices" (like the evtchn one)?> I think the front/back xenbus and device drivers are still a bit of a > mess with regard to modules; I have to admit I tend to just compile them > all in. It would be nice to get all that properly sorted out so they > can be sensibly modularized and auto-loaded as required. (A good first > step would be to systematically work out what compile combinations > actually work or fail, hint, hint ;)AFAIK the frontend driver modules work correctly as modules and distros have been compiling them modular for a while (Debian certainly has). The situation is certainly not as rosy for *back. It''d certainly be nice if some (or all) of the xenbus stuff could become modular too although I would guess there will be a core which must be compiled in. Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Mar-30 14:22 UTC
Re: [Xen-devel] Automatic loading of xen-evtchn module in Xen 4.0.0?
On Tue, Mar 30, 2010 at 10:44:57AM +0100, Ian Campbell wrote:> On Sat, 2010-03-27 at 22:18 +0000, Jeremy Fitzhardinge wrote: > > On 03/27/2010 10:32 AM, Pasi Kärkkäinen wrote: > > > Hello, > > > > > > xend automatically mounts /proc/xen, but it doesn''t try loading evtchn or xen-evtchn modules. > > > Should we add automatic loading of these modules? > > > > > > xend doesn''t start if xen event channel functionality is missing. > > > > > > Fedora Xen rpms do load modules automatically, like this: > > > > > > # cat /etc/sysconfig/modules/xen.modules > > > #!/bin/sh > > > > > > test -d /proc/xen || exit 0 > > > test -f /proc/xen/capabilities || mount -t xenfs xen /proc/xen > > > test -f /proc/xen/capabilities || exit 0 > > > grep -q "control_d" /proc/xen/capabilities || exit 0 > > > > > > for module in xen-evtchn evtchn blkbk netbk; do > > > modprobe ${module}>/dev/null 2>&1 > > > done > > > > > > > We should be able to make these mount automatically with udev rules, I > > think, but I''m not sure whether you can do udev rules to mount misc > > devices (which is what /dev/xen/evtchn is). > > I think you mean "modprobe misc devices" not mount. > > I see various uses of the MODULE_ALIAS_MISCDEV macro in the kernel -- > but I''m not sure what actually causes a matching hotplug event to be > generated, other than trying to open a matching device node which with > udev won''t exist until the module is loaded.One idea that I submitted to dracut mailing list (the initramfs creation/running tool under FC12 and above) was to leverage ''xen-detect'' and load the appropiate modules. This way during bootup, bamm, you get all your modules in. But they righly suggested that there should be an uevent to kick udev to start loading them and use the MODULE_ALIAS. But to use MODULE_ALIAS, you have to get an uevent, and to get an uevent, you have to load the module, so it looks to be a chicken-and-egg problem.> > Perhaps something in the xen core code (or xenbus?) could generate the > necessary hotplug events for the Xen "platform devices" (like the evtchn > one)?But I think that comes back to the chicken-and-egg. If you have all of the front-end drivers compiled as modules, how would they (front-end drivers) generate the hotplug events when they are not loaded? Perhaps have the xenbus try to load the modules if it detects that is running under Xen? Something similar to how the kernel tries to mount various filesystem modules when mounting a disk? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2010-Mar-30 14:26 UTC
Re: [Xen-devel] Automatic loading of xen-evtchn module in Xen 4.0.0?
On Tue, 2010-03-30 at 15:22 +0100, Konrad Rzeszutek Wilk wrote:> On Tue, Mar 30, 2010 at 10:44:57AM +0100, Ian Campbell wrote: > > On Sat, 2010-03-27 at 22:18 +0000, Jeremy Fitzhardinge wrote: > > > On 03/27/2010 10:32 AM, Pasi Kärkkäinen wrote: > > > > Hello, > > > > > > > > xend automatically mounts /proc/xen, but it doesn''t try loading evtchn or xen-evtchn modules. > > > > Should we add automatic loading of these modules? > > > > > > > > xend doesn''t start if xen event channel functionality is missing. > > > > > > > > Fedora Xen rpms do load modules automatically, like this: > > > > > > > > # cat /etc/sysconfig/modules/xen.modules > > > > #!/bin/sh > > > > > > > > test -d /proc/xen || exit 0 > > > > test -f /proc/xen/capabilities || mount -t xenfs xen /proc/xen > > > > test -f /proc/xen/capabilities || exit 0 > > > > grep -q "control_d" /proc/xen/capabilities || exit 0 > > > > > > > > for module in xen-evtchn evtchn blkbk netbk; do > > > > modprobe ${module}>/dev/null 2>&1 > > > > done > > > > > > > > > > We should be able to make these mount automatically with udev rules, I > > > think, but I''m not sure whether you can do udev rules to mount misc > > > devices (which is what /dev/xen/evtchn is). > > > > I think you mean "modprobe misc devices" not mount. > > > > I see various uses of the MODULE_ALIAS_MISCDEV macro in the kernel -- > > but I''m not sure what actually causes a matching hotplug event to be > > generated, other than trying to open a matching device node which with > > udev won''t exist until the module is loaded. > > One idea that I submitted to dracut mailing list (the initramfs > creation/running tool under FC12 and above) was to leverage ''xen-detect'' > and load the appropiate modules. This way during bootup, bamm, you get > all your modules in. > > But they righly suggested that there should be an uevent to kick udev to > start loading them and use the MODULE_ALIAS. But to use MODULE_ALIAS, > you have to get an uevent, and to get an uevent, you have to load the > module, so it looks to be a chicken-and-egg problem. > > > > > Perhaps something in the xen core code (or xenbus?) could generate the > > necessary hotplug events for the Xen "platform devices" (like the evtchn > > one)? > > But I think that comes back to the chicken-and-egg. If you have all of > the front-end drivers compiled as modules, how would they (front-end > drivers) generate the hotplug events when they are not loaded?The xenbus infrastructure sees e.g. the /vif/ node appear in xenstore and generates an event for "xen:vif" which xen-netfront declares as an alias so the module will be loaded. In other words it is up to the _bus_ implementation not the drivers to generate the events to cause the drivers to be loaded. For frontend drivers this works today, AFAIK. The issue with the evtchn driver is that there is no bus as such. We could potentially make use of the existing platform device infrastructure which is kind of like a bus for system devices which aren''t really on a bus. I guess the xenbus driver itself falls into the same sort of category as the evtchn driver. Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-Mar-30 22:47 UTC
Re: [Xen-devel] Automatic loading of xen-evtchn module in Xen 4.0.0?
On 03/30/2010 02:44 AM, Ian Campbell wrote:> On Sat, 2010-03-27 at 22:18 +0000, Jeremy Fitzhardinge wrote: > >> On 03/27/2010 10:32 AM, Pasi Kärkkäinen wrote: >> >>> Hello, >>> >>> xend automatically mounts /proc/xen, but it doesn''t try loading evtchn or xen-evtchn modules. >>> Should we add automatic loading of these modules? >>> >>> xend doesn''t start if xen event channel functionality is missing. >>> >>> Fedora Xen rpms do load modules automatically, like this: >>> >>> # cat /etc/sysconfig/modules/xen.modules >>> #!/bin/sh >>> >>> test -d /proc/xen || exit 0 >>> test -f /proc/xen/capabilities || mount -t xenfs xen /proc/xen >>> test -f /proc/xen/capabilities || exit 0 >>> grep -q "control_d" /proc/xen/capabilities || exit 0 >>> >>> for module in xen-evtchn evtchn blkbk netbk; do >>> modprobe ${module}>/dev/null 2>&1 >>> done >>> >>> >> We should be able to make these mount automatically with udev rules, I >> think, but I''m not sure whether you can do udev rules to mount misc >> devices (which is what /dev/xen/evtchn is). >> > I think you mean "modprobe misc devices" not mount. >Yep.> I see various uses of the MODULE_ALIAS_MISCDEV macro in the kernel -- > but I''m not sure what actually causes a matching hotplug event to be > generated, other than trying to open a matching device node which with > udev won''t exist until the module is loaded. > > Perhaps something in the xen core code (or xenbus?) could generate the > necessary hotplug events for the Xen "platform devices" (like the evtchn > one)? > > >> I think the front/back xenbus and device drivers are still a bit of a >> mess with regard to modules; I have to admit I tend to just compile them >> all in. It would be nice to get all that properly sorted out so they >> can be sensibly modularized and auto-loaded as required. (A good first >> step would be to systematically work out what compile combinations >> actually work or fail, hint, hint ;) >> > AFAIK the frontend driver modules work correctly as modules and distros > have been compiling them modular for a while (Debian certainly has). >Yes, I think frontend drivers are OK. But the various xenbus bits seem a bit fragile.> The situation is certainly not as rosy for *back. It''d certainly be nice > if some (or all) of the xenbus stuff could become modular too although I > would guess there will be a core which must be compiled in. >In practice there''s no need to get very modular. There''s some value in not loading all that backend support code unless we''re actually dom0, but I don''t think there''s much value in getting too fine-grained beyond that. The actual drivers themselves are fairly ordinary, so they will probably just work if we can make xenbus generate the right request events. Hm, I guess that would be the tool stack setting things up for a new domain, and requesting backend devices in the process, no? But it doesn''t help for things like evtchn and gntdev. But there''s no huge burden in just making the toolstack startup scripts do modprobes for them, just as it should be mounting /proc/xen. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2010-Mar-31 08:40 UTC
Re: [Xen-devel] Automatic loading of xen-evtchn module in Xen 4.0.0?
On Tue, 2010-03-30 at 23:47 +0100, Jeremy Fitzhardinge wrote:> > In practice there''s no need to get very modular. There''s some value > in > not loading all that backend support code unless we''re actually dom0, > but I don''t think there''s much value in getting too fine-grained > beyond that. The actual drivers themselves are fairly ordinary, so > they will probably just work if we can make xenbus generate the right > request events. Hm, I guess that would be the tool stack setting > things up for a new domain, and requesting backend devices in the > process, no?What happens with frontend hotplug, and should happen with backends too, is that the xenbus core watches the "device" (or "backend") node in xenstore (relative to the domain''s root path "/local/domain/<X>/"). The tools then create the necessary nodes to create the device which triggers the xenstore watch, e.g. on a path starting with "vif", which causes xenbus to generate a uevent with the necessary MODALIAS, see drivers/xen/xenbus/xenbus_probe.c:xenbus_uevent(). The only problem with this is getting the initial xenbus-backend.ko loaded so that it can do the watch -- perhaps that falls in the same boat as evtchn and gntdev etc. Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Mar-31 13:39 UTC
Re: [Xen-devel] Automatic loading of xen-evtchn module in Xen 4.0.0?
On Wed, Mar 31, 2010 at 09:40:40AM +0100, Ian Campbell wrote:> On Tue, 2010-03-30 at 23:47 +0100, Jeremy Fitzhardinge wrote: > > > > In practice there''s no need to get very modular. There''s some value > > in > > not loading all that backend support code unless we''re actually dom0, > > but I don''t think there''s much value in getting too fine-grained > > beyond that. The actual drivers themselves are fairly ordinary, so > > they will probably just work if we can make xenbus generate the right > > request events. Hm, I guess that would be the tool stack setting > > things up for a new domain, and requesting backend devices in the > > process, no? > > What happens with frontend hotplug, and should happen with backends too, > is that the xenbus core watches the "device" (or "backend") node in > xenstore (relative to the domain''s root path "/local/domain/<X>/"). The > tools then create the necessary nodes to create the device which > triggers the xenstore watch, e.g. on a path starting with "vif", which > causes xenbus to generate a uevent with the necessary MODALIAS, see > drivers/xen/xenbus/xenbus_probe.c:xenbus_uevent(). > > The only problem with this is getting the initial xenbus-backend.ko > loaded so that it can do the watch -- perhaps that falls in the same > boat as evtchn and gntdev etc... which goes back to what I suggested some time ago, that might do the trick. That is to make the owner of /sys/hypervisor do a kobject_uevent and have an udev rule that would listen to this and load the xenbus-backend.ko. That should start the avalanche of Xen backend module loading, I think? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel