Hi guys, I''m trying to understand the architecture of the PCI pass-through functionality on Xen. Just to be clear, I''m (for now) trying to understand what''s happening when a PCI device is "hidden" from (PV) Dom0 and exposed to a PV DomU. I was originally assuming that as long as Dom0 was told not to touch a particular PCI device, then DomU could be granted direct access do it, and operations on that device would go through its normal drivers on DomU, and the DomU kernel with no involvement whatsoever from Dom0. The model in my mind is that since Dom0 and DomU are equally paravirtualized, there is no reason that DomU couldn''t be given the same exact mechanism for access to the PCI device as Dom0 is normally given...it''s just a matter of configuration. But in looking in how to configure this, I see there are pciback and pcifront drivers...just like netback and netfront. I thought maybe pciback''s sole purpose was to hide specified PCI devices from Dom0, but there is quite a bit of code in the source for pciback. Also in pcifront I see xenbus.c, suggesting it communicates with its back-end counterpart just like netfront or blkfront. So, when one grant''s "direct" access to a PCI device to DomU, is it still being piped through Dom0 like block devices and interfaces? If so why is this necessary? Does the device-specific driver run in Dom0 or DomU? Thanks, Dave _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi Dave,> Hi guys, I''m trying to understand the architecture of the PCI > pass-through functionality on Xen. Just to be clear, I''m (for now) > trying to understand what''s happening when a PCI device is "hidden" > from (PV) Dom0 and exposed to a PV DomU. > > I was originally assuming that as long as Dom0 was told not to touch a > particular PCI device, then DomU could be granted direct access do it, > and operations on that device would go through its normal drivers on > DomU, and the DomU kernel with no involvement whatsoever from Dom0.That''s almost the case :-)> The model in my mind is that since Dom0 and DomU are equally > paravirtualized, there is no reason that DomU couldn''t be given the > same exact mechanism for access to the PCI device as Dom0 is normally > given...it''s just a matter of configuration.Again, it''s almost like that. Logically, it pretty much is but there are a few practical differences.> But in looking in how to configure this, I see there are pciback and > pcifront drivers...just like netback and netfront. I thought maybe > pciback''s sole purpose was to hide specified PCI devices from Dom0,I believe that''s one of it''s functions: grabbing PCI devices on behalf of the domU so that real device drivers in dom0 won''t grab them and start using them.> but there is quite a bit of code in the source for pciback. Also in > pcifront I see xenbus.c, suggesting it communicates with its back-end > counterpart just like netfront or blkfront. > > So, when one grant''s "direct" access to a PCI device to DomU, is it > still being piped through Dom0 like block devices and interfaces? If > so why is this necessary? Does the device-specific driver run in Dom0 > or DomU?The "fast path" should go direct. But certain operations need to go via dom0. Basically these are the operations that are logically handled by the PCI bus driver rather than the specific device driver. So, for instance, if you want to access PCI configuration space you usually do that by calling into the PCI bus driver from your PCI device driver. The bus driver will then do the right thing, depending on your hardware setup. The pciback / pcifront drivers provide the mechanism to do operations on config space and to expose only relevant device configuration information to the domU. There are also access controls on what the domU is allowed to do to config space to help avoid it causing system-wide problems. There may also be some correctness concerns which would make concurrently executing bus drivers talking to the hardware unsafe. The actual talking to the PCI device does generally happen directly, via IO memory regions, IO ports, in-memory descriptor rings etc. Cheers, Mark -- Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel! _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
They''re not really equal -- dom0 manages the platform (IRQ routing, ACPI OSPM stuff, etc.). This type of thing is hard to decentralise, so dom0 continues to manage basic setup of PCI devices, some of which is triggered by accesses to PCI config space by domUs, via pcifront/pciback. However, device-specific management is done by the domU itself. dom0 is not involved in any data paths. -- Keir On 6/11/07 20:27, "David Stone" <unclestoner@gmail.com> wrote:> Hi guys, I''m trying to understand the architecture of the PCI > pass-through functionality on Xen. Just to be clear, I''m (for now) > trying to understand what''s happening when a PCI device is "hidden" > from (PV) Dom0 and exposed to a PV DomU. > > I was originally assuming that as long as Dom0 was told not to touch a > particular PCI device, then DomU could be granted direct access do it, > and operations on that device would go through its normal drivers on > DomU, and the DomU kernel with no involvement whatsoever from Dom0. > The model in my mind is that since Dom0 and DomU are equally > paravirtualized, there is no reason that DomU couldn''t be given the > same exact mechanism for access to the PCI device as Dom0 is normally > given...it''s just a matter of configuration. > > But in looking in how to configure this, I see there are pciback and > pcifront drivers...just like netback and netfront. I thought maybe > pciback''s sole purpose was to hide specified PCI devices from Dom0, > but there is quite a bit of code in the source for pciback. Also in > pcifront I see xenbus.c, suggesting it communicates with its back-end > counterpart just like netfront or blkfront. > > So, when one grant''s "direct" access to a PCI device to DomU, is it > still being piped through Dom0 like block devices and interfaces? If > so why is this necessary? Does the device-specific driver run in Dom0 > or DomU? > > Thanks, > Dave > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Thanks for your responses Mark and Keir, that makes sense to me. Dave On Nov 7, 2007 2:31 AM, Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:> They''re not really equal -- dom0 manages the platform (IRQ routing, ACPI > OSPM stuff, etc.). This type of thing is hard to decentralise, so dom0 > continues to manage basic setup of PCI devices, some of which is triggered > by accesses to PCI config space by domUs, via pcifront/pciback. However, > device-specific management is done by the domU itself. dom0 is not involved > in any data paths. > > -- Keir > > > On 6/11/07 20:27, "David Stone" <unclestoner@gmail.com> wrote: > > > Hi guys, I''m trying to understand the architecture of the PCI > > pass-through functionality on Xen. Just to be clear, I''m (for now) > > trying to understand what''s happening when a PCI device is "hidden" > > from (PV) Dom0 and exposed to a PV DomU. > > > > I was originally assuming that as long as Dom0 was told not to touch a > > particular PCI device, then DomU could be granted direct access do it, > > and operations on that device would go through its normal drivers on > > DomU, and the DomU kernel with no involvement whatsoever from Dom0. > > The model in my mind is that since Dom0 and DomU are equally > > paravirtualized, there is no reason that DomU couldn''t be given the > > same exact mechanism for access to the PCI device as Dom0 is normally > > given...it''s just a matter of configuration. > > > > But in looking in how to configure this, I see there are pciback and > > pcifront drivers...just like netback and netfront. I thought maybe > > pciback''s sole purpose was to hide specified PCI devices from Dom0, > > but there is quite a bit of code in the source for pciback. Also in > > pcifront I see xenbus.c, suggesting it communicates with its back-end > > counterpart just like netfront or blkfront. > > > > So, when one grant''s "direct" access to a PCI device to DomU, is it > > still being piped through Dom0 like block devices and interfaces? If > > so why is this necessary? Does the device-specific driver run in Dom0 > > or DomU? > > > > Thanks, > > Dave > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel