Ryan, This is great news, thanks a lot. A couple of comments:> slots - lists all of the PCI slots that > the PCI backend will try to seizeStylistically I generally expect sysfs files to only contain one value. So I would have maybe expected a file per registered slot... People closer to the kernel community may have a more definitive answer... This is really just a nitpick, but the kernel folks are quite picky about what they allow into sysfs.> new_slot - write the name of a slot here > (in 0000:00:00.0 format) to have the PCI > Backend seize the device in this slot > remove_slot - write the name of a slot here to have > the PCI Backend no longer try to seize a > device in this slotIf you did have a file per slot, you could drop remove_slot and echo "remove" into the file (or zero, or have a directory per slot and a "remove" file underneath it - you get the idea...).> Note that writing to new_slot/remove_slot does not actually change > whether the PCI Backend is actually bound to the device in that slot or > not. Instead, it tells the PCI backend which slots it should be > interested in. The sysfs attributes "bind" and "unbind" (which are > common to all drivers, not just the PCI Backend) must be used to > actually add or remove a device from the PCI backend driver. Note that > the syntax for specifying a device to bind and unbind is very strict (do > not append a newline).This stuff is super cool. It''s amazingly flexible - better than we ever had before. Do you think you might be able to integrate the bind / unbind operations into pciif.py in Xend? That way, users don''t need to worry about reassigning device drivers or clearing them up afterwards, they could just (for instance) put the slot number in their config file. From there it''s a fairly small step to imagine a GUI where you just drop a NIC from a list onto a virtual machine config :-) Really cool stuff, anyhow, congratulations. Cheers, Mark -- Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel! _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Attached to this e-mail are patches which provide support for late
binding of PCI devices to the PCI backend driver in dom0. This allows
you to bind devices to the backend driver *after* dom0 has booted. You
are no longer required to specify the devices to hide on the kernel
command-line. Using the bind/unbind driver attributes
(see /sys/bus/pci/drivers/pciback), you can specify which devices that
the PCI backend will seize. There are three new driver attributes in
that directory:
slots - lists all of the PCI slots that
the PCI backend will try to seize
new_slot - write the name of a slot here
(in 0000:00:00.0 format) to have the PCI
Backend seize the device in this slot
remove_slot - write the name of a slot here to have
the PCI Backend no longer try to seize a
device in this slot
Note that writing to new_slot/remove_slot does not actually change
whether the PCI Backend is actually bound to the device in that slot or
not. Instead, it tells the PCI backend which slots it should be
interested in. The sysfs attributes "bind" and "unbind"
(which are
common to all drivers, not just the PCI Backend) must be used to
actually add or remove a device from the PCI backend driver. Note that
the syntax for specifying a device to bind and unbind is very strict (do
not append a newline).
For Example:
# Add a new slot to the PCI Backend''s list
echo -n 0000:01:04.d > /sys/bus/pci/drivers/pciback/new_slot
# Now that the backend is watching for the slot, bind to it
echo -n 0000:01:04.d > /sys/bus/pci/drivers/pciback/bind
# Unbind a PCI network card from its network driver
echo -n 0000:05:02.0 > /sys/bus/pci/drivers/3c905/unbind
# And now bind it to the PCI Backend
echo -n 0000:05:02.0 > /sys/bus/pci/drivers/pciback/new_slot
echo -n 0000:05:02.0 > /sys/bus/pci/drivers/pciback/bind
Unfortunately, Linux makes it possible to remove (unbind) a PCI device
from the PCI backend while that device is attached to a driver domain.
It is also possible to unload the PCI Backend module while a PCI
Frontend is attached. DON''T DO EITHER OF THESE ACTIONS. This patch will
output warnings if you do try and do these. Be aware that while access
to the configuration space of the device has been revoked, the driver
domain can still access the I/O resources of the device as they have not
been revoked (although I *hope* to explore adding support for this
soon). Before unloading the module or unbinding a device, shutdown your
driver domain.
These patches also convert a few function and variable declarations to
static (no sense in polluting the global namespace with local function
names) and rename a few structures in drivers/xen/pciback/pci_stub.c.
device_bind.patch patches a Linux bug in the driver core in regards to
the bind sysfs driver attribute. I''ve submitted this to lkml, but it
should be included in Xen now for people who will use this late binding
capability.
Signed-off-by: Ryan Wilson <hap9@epoch.ncsc.mil>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
On 23 Mar 2006, at 00:32, Ryan wrote:> # Add a new slot to the PCI Backend''s list > echo -n 0000:01:04.d > /sys/bus/pci/drivers/pciback/new_slot > # Now that the backend is watching for the slot, bind to it > echo -n 0000:01:04.d > /sys/bus/pci/drivers/pciback/bindThanks for the patches, which I''ve now applied. But why the two-stage process to bind to a PCI device? Does writing to ''bind'' get swallowed by sysfs and not make it to pciback unless you''ve already registered interest with sysfs? -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> # Add a new slot to the PCI Backend''s list > echo -n 0000:01:04.d > /sys/bus/pci/drivers/pciback/new_slot > # Now that the backend is watching for the slot, bind to it > echo -n 0000:01:04.d > /sys/bus/pci/drivers/pciback/bindThanks for the patches, which I''ve now applied. Why do you need the two-stage process to bind on a PCI device? Is it because writing to the ''bind'' file gets swallowed by sysfs unless your driver has pre-registered an interest in the particular PCI device? -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, 2006-03-23 at 10:00 +0000, Keir Fraser wrote:> On 23 Mar 2006, at 00:32, Ryan wrote: > > > # Add a new slot to the PCI Backend''s list > > echo -n 0000:01:04.d > /sys/bus/pci/drivers/pciback/new_slot > > # Now that the backend is watching for the slot, bind to it > > echo -n 0000:01:04.d > /sys/bus/pci/drivers/pciback/bind > > Thanks for the patches, which I''ve now applied. But why the two-stage > process to bind to a PCI device? Does writing to ''bind'' get swallowed > by sysfs and not make it to pciback unless you''ve already registered > interest with sysfs?The bind attribute calls the probe handler on the device driver with the specified device. Unfortunately, there''s no way for me (at least, not that I''m aware of) to detect in my probe routine if the user is wanting to forcibly bind the device to the backend or if it''s a normal probe from a device insertion or driver registration. This same problem exists for existing drivers (see http://lwn.net/Articles/143397/ ) where the driver must want to bind to the device. The driver core people expected this and added an attribute whereby a user can tell a driver to be interested in other device ids through the new_id driver attribute (PCI class, vendor, device, etc.). But since the PCI backend registers that it is interested in *every* device (so that it can check the slot number instead of any kind of device ID), I had to add those new attributes (new_slot/remove_slot) which serve the same purpose as new_id but are specific to the needs of the PCI backend. It would probably be possible to have a call to new_slot do the binding as well (like call pci_find_device or something similar and then driver_attach), but I figured since the sysfs infrastructure already did it for me, I wouldn''t duplicate that code. In addition, you may wish to add/remove slots for devices which aren''t inserted in the system yet (they would get automatically bound when they are inserted through the normal probing that occurs). I''m open to suggestions for a better way, but I thought this achieved the desired functionality while closely matching the kernel''s usual interface (my new_slot/remove_slot matches the purpose of the new_id sysfs attribute). Ryan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wed, 2006-03-22 at 20:41 +0000, Mark Williamson wrote:> Ryan, > > This is great news, thanks a lot.Your welcome! :)> > A couple of comments: > > > slots - lists all of the PCI slots that > > the PCI backend will try to seize > > Stylistically I generally expect sysfs files to only contain one value. So I > would have maybe expected a file per registered slot... People closer to the > kernel community may have a more definitive answer... >I think the intention was to have only values of one type, not necessarily just one value. For example, the resource attribute on PCI devices contains a line for every resource associated with that PCI device. From ''Documentation/filesystems/sysfs.txt'': "Attributes should be ASCII text files, preferably with only one value per file. It is noted that it may not be efficient to contain only value per file, so it is socially acceptable to express an array of values of the same type." I think the slots file is acceptable, but if not, it can be changed.> This stuff is super cool. It''s amazingly flexible - better than we ever had > before. Do you think you might be able to integrate the bind / unbind > operations into pciif.py in Xend? That way, users don''t need to worry about > reassigning device drivers or clearing them up afterwards, they could just > (for instance) put the slot number in their config file. From there it''s a > fairly small step to imagine a GUI where you just drop a NIC from a list onto > a virtual machine config :-) >I''m not 100% sure I understand your suggestion here, but I don''t know that it would be appropriate for xend to automatically unbind a device from another driver (where it may be in-use) if the user specified it in the configuration file. I think it would be better to fail and make the user manually unbind it from the other driver. Although it would certainly be nice to have the PCI backend automatically pick up a device that''s requested if it''s not attached to any other driver (and allow the user to skip the bind step). I won''t make any promises about writing code to do this as I''ve got a few other things that I''m working on right now, but I''ll keep it in mind. Ryan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel