Xen devel - Aug 2005 - frontend and backend devices and different types of hw - pci for example

Hello, 

 In the sparse tree , under drivers/xen, there are frontend and backend drivers
for network (netback and netfront), for block devices (blkfront and
blkback), and for
usb devices (usbfront and usbback). 
What about other devices ? let''s say a PCI sound card (or any other PCI
device).
 Where is the software that should handle it ? I remember I saw somewhere some
discussion about PCI configuration space, but I don''t remember where.

TIA,
Sting

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> What about other devices ? let''s say a PCI sound card (or any
other PCI
> device). Where is the software that should handle it ? I remember I saw
> somewhere some discussion about PCI configuration space, but I
don''t
> remember where.
That code is in Xen itself in Xen 2.0.  Xen controls access to the PCI 
configuration spaces so that guests can only see the devices they have access 
to.  It also controls the IO memory / ports that domains are allowed to 
access in order to control PCI devices.

Note that giving direct physical access to a PCI device has security 
implications since the guest can potentially use the cards'' DMA
capabilities
to access all of physical memory.  The front/back-style devices do not have 
this limitation.

Btw, I''ve laid some groundwork for a virtual sound device but
haven''t had much
time to hack on it yet.

Cheers,
Mark

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hello, 

 I had looked at the code of 2.0.*  under xen/arch/x86 saw 
pci-irq.c and pci-pc.c and pci-x86.c which as I understand handle pci devices 
other than net/usb.
However, I did not saw such modules in the unstable version. 
May I ask : is this PCI support for non net/usb PCI devices  removed
(or temporarily removed) from the unstable version? or maybe I simply
missed it ?

>Note that giving direct physical access to a PCI device has security
>implications since the guest can potentially use the cards'' DMA
capabilities
>to access all of physical memory.  
Will IOMMU support help solving this security problems ? 


Regards,
Sting

On 8/28/05, Mark Williamson <mark.williamson@cl.cam.ac.uk>
wrote:
> > What about other devices ? let''s say a PCI sound card (or any
other PCI
> > device). Where is the software that should handle it ? I remember I
saw
> > somewhere some discussion about PCI configuration space, but I
don''t
> > remember where.
> 
> That code is in Xen itself in Xen 2.0.  Xen controls access to the PCI
> configuration spaces so that guests can only see the devices they have
access
> to.  It also controls the IO memory / ports that domains are allowed to
> access in order to control PCI devices.
> 
> Note that giving direct physical access to a PCI device has security
> implications since the guest can potentially use the cards'' DMA
capabilities
> to access all of physical memory.  The front/back-style devices do not have
> this limitation.
> 
> Btw, I''ve laid some groundwork for a virtual sound device but
haven''t had much
> time to hack on it yet.
> 
> Cheers,
> Mark
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 29 Aug 2005, at 07:57, Sting Zax wrote:
>  I had looked at the code of 2.0.*  under xen/arch/x86 saw
> pci-irq.c and pci-pc.c and pci-x86.c which as I understand handle pci 
> devices
> other than net/usb.
> However, I did not saw such modules in the unstable version.
> May I ask : is this PCI support for non net/usb PCI devices  removed
> (or temporarily removed) from the unstable version? or maybe I simply
> missed it ?
Xen no longer controls PCI devices -- the PCI code in domain 0 now 
accesses PCI devices directly. You can run pretty much any device 
driver you want in domain 0.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>  I had looked at the code of 2.0.*  under xen/arch/x86 saw
> pci-irq.c and pci-pc.c and pci-x86.c which as I understand handle pci
> devices other than net/usb.
> However, I did not saw such modules in the unstable version.
> May I ask : is this PCI support for non net/usb PCI devices  removed
> (or temporarily removed) from the unstable version? or maybe I simply
> missed it ?
Dom0 now basically owns anything to do with PCI whereas previously Xen 
controlled core PCI stuff and dom0 just accessed devices.  Support for giving 
device access to other domains will need to be implemented in dom0 (this 
hasn''t been done yet but it''s hoped the feature will be back
in time for the
release).
> >Note that giving direct physical access to a PCI device has security
> >implications since the guest can potentially use the cards''
DMA
> > capabilities to access all of physical memory.
>
> Will IOMMU support help solving this security problems ?
Yes but only if it enforces access permissions fully i.e. I don''t think
the
IOEMU in AMD64 machines is sufficient.  From the looks of Pacifica it might 
have sufficient support to control the DMA problem, I''m sure Intel have
a
similar solution (although I don''t think it''s implemented in
Vanderpool -
they''ll probably need chipset support).

Cheers,
Mark
>
> Regards,
> Sting
>
> On 8/28/05, Mark Williamson <mark.williamson@cl.cam.ac.uk> wrote:
> > > What about other devices ? let''s say a PCI sound card
(or any other PCI
> > > device). Where is the software that should handle it ? I remember
I saw
> > > somewhere some discussion about PCI configuration space, but I
don''t
> > > remember where.
> >
> > That code is in Xen itself in Xen 2.0.  Xen controls access to the PCI
> > configuration spaces so that guests can only see the devices they have
> > access to.  It also controls the IO memory / ports that domains are
> > allowed to access in order to control PCI devices.
> >
> > Note that giving direct physical access to a PCI device has security
> > implications since the guest can potentially use the cards''
DMA
> > capabilities to access all of physical memory.  The front/back-style
> > devices do not have this limitation.
> >
> > Btw, I''ve laid some groundwork for a virtual sound device but
haven''t had
> > much time to hack on it yet.
> >
> > Cheers,
> > Mark
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

xen-devel-bounces@lists.xensource.com wrote on 08/29/2005 06:45:51 AM:
> >  I had looked at the code of 2.0.*  under xen/arch/x86 saw
> > pci-irq.c and pci-pc.c and pci-x86.c which as I understand handle pci
> > devices other than net/usb.
> > However, I did not saw such modules in the unstable version.
> > May I ask : is this PCI support for non net/usb PCI devices  removed
> > (or temporarily removed) from the unstable version? or maybe I simply
> > missed it ?
> 
> Dom0 now basically owns anything to do with PCI whereas previously Xen 
> controlled core PCI stuff and dom0 just accessed devices.  Support for 
giving 
> device access to other domains will need to be implemented in dom0 (this 
> hasn''t been done yet but it''s hoped the feature will be
back in time for
the 
> release).
Do you think that an emulated PCI layer underneath every domU could be a 
possibility for a solution of moving PCI devices to user domains? I have 
had some success with it and got as far as for example moving a PCI 
ethernet card or the whole USB controller to a user domain and making the 
user domain kernel activate its drivers. I did this by reading the PCI 
config space (256 bytes) from the device and presenting the data to the 
user level in the emulated PCI bus. However I have then encountered a 
couple of problems afterwards when trying to activate the IRQ. There seems 
to be some translation going on of a PCI IRQ number to the actual number 
the system is using (due to APIC I suppose) and so the data exchange with 
the device did not start.

  Stefan
> 
> > >Note that giving direct physical access to a PCI device has
security
> > >implications since the guest can potentially use the
cards'' DMA
> > > capabilities to access all of physical memory.
> >
> > Will IOMMU support help solving this security problems ?
> 
> Yes but only if it enforces access permissions fully i.e. I don''t
think
the 
> IOEMU in AMD64 machines is sufficient.  From the looks of Pacifica it 
might 
> have sufficient support to control the DMA problem, I''m sure Intel
have
a 
> similar solution (although I don''t think it''s implemented
in Vanderpool
- 
> they''ll probably need chipset support).
> 
> Cheers,
> Mark
> 
> >
> > Regards,
> > Sting
> >
> > On 8/28/05, Mark Williamson <mark.williamson@cl.cam.ac.uk>
wrote:
> > > > What about other devices ? let''s say a PCI sound
card (or any
other PCI
> > > > device). Where is the software that should handle it ? I
remember
I saw
> > > > somewhere some discussion about PCI configuration space, but
I
don''t
> > > > remember where.
> > >
> > > That code is in Xen itself in Xen 2.0.  Xen controls access to
the
PCI
> > > configuration spaces so that guests can only see the devices they
have
> > > access to.  It also controls the IO memory / ports that domains
are
> > > allowed to access in order to control PCI devices.
> > >
> > > Note that giving direct physical access to a PCI device has
security
> > > implications since the guest can potentially use the
cards'' DMA
> > > capabilities to access all of physical memory.  The
front/back-style
> > > devices do not have this limitation.
> > >
> > > Btw, I''ve laid some groundwork for a virtual sound
device but
haven''t had
> > > much time to hack on it yet.
> > >
> > > Cheers,
> > > Mark
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> Do you think that an emulated PCI layer underneath every domU could be a
> possibility for a solution of moving PCI devices to user domains? I have
> had some success with it and got as far as for example moving a PCI
> ethernet card or the whole USB controller to a user domain and making the
> user domain kernel activate its drivers.
Possibly...  I would have been inclined to do it using some sort of 
interdomain communication rather than using an emulator in Xen but I''m 
actually open to persuasion that I''m wrong on this point ;-)
> I did this by reading the PCI 
> config space (256 bytes) from the device and presenting the data to the
> user level in the emulated PCI bus. However I have then encountered a
> couple of problems afterwards when trying to activate the IRQ. There seems
> to be some translation going on of a PCI IRQ number to the actual number
> the system is using (due to APIC I suppose) and so the data exchange with
> the device did not start.
Thanks for the extra detail.

Hmmm.  Sounds a bit weird...  Afraid I''m not sure what might be going
on here.

Cheers,
Mark
>   Stefan
>
> > > >Note that giving direct physical access to a PCI device has
security
> > > >implications since the guest can potentially use the
cards'' DMA
> > > > capabilities to access all of physical memory.
> > >
> > > Will IOMMU support help solving this security problems ?
> >
> > Yes but only if it enforces access permissions fully i.e. I
don''t think
>
> the
>
> > IOEMU in AMD64 machines is sufficient.  From the looks of Pacifica it
>
> might
>
> > have sufficient support to control the DMA problem, I''m sure
Intel have
>
> a
>
> > similar solution (although I don''t think it''s
implemented in Vanderpool
>
> -
>
> > they''ll probably need chipset support).
> >
> > Cheers,
> > Mark
> >
> > > Regards,
> > > Sting
> > >
> > > On 8/28/05, Mark Williamson <mark.williamson@cl.cam.ac.uk>
wrote:
> > > > > What about other devices ? let''s say a PCI
sound card (or any
>
> other PCI
>
> > > > > device). Where is the software that should handle it ?
I remember
>
> I saw
>
> > > > > somewhere some discussion about PCI configuration
space, but I
>
> don''t
>
> > > > > remember where.
> > > >
> > > > That code is in Xen itself in Xen 2.0.  Xen controls access
to the
>
> PCI
>
> > > > configuration spaces so that guests can only see the devices
they
>
> have
>
> > > > access to.  It also controls the IO memory / ports that
domains are
> > > > allowed to access in order to control PCI devices.
> > > >
> > > > Note that giving direct physical access to a PCI device has
security
> > > > implications since the guest can potentially use the
cards'' DMA
> > > > capabilities to access all of physical memory.  The
front/back-style
> > > > devices do not have this limitation.
> > > >
> > > > Btw, I''ve laid some groundwork for a virtual sound
device but
>
> haven''t had
>
> > > > much time to hack on it yet.
> > > >
> > > > Cheers,
> > > > Mark
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

xen-devel-bounces@lists.xensource.com wrote on 09/03/2005 01:46:24 PM:
> > Do you think that an emulated PCI layer underneath every domU could be
a
> > possibility for a solution of moving PCI devices to user domains? I 
have
> > had some success with it and got as far as for example moving a PCI
> > ethernet card or the whole USB controller to a user domain and making 
the
> > user domain kernel activate its drivers.
> 
> Possibly...  I would have been inclined to do it using some sort of 
> interdomain communication rather than using an emulator in Xen but
I''m
> actually open to persuasion that I''m wrong on this point ;-)
I thought about interdomain communication for emulating PCI devices. It 
''feels'' like this would rip apart both  domain''s PCI
layers quite a bit.
> 
> > I did this by reading the PCI 
> > config space (256 bytes) from the device and presenting the data to 
the
> > user level in the emulated PCI bus. However I have then encountered a
I wrote this wrong: I am not presenting this to the user level, but to the 
user domain kernel.
> > couple of problems afterwards when trying to activate the IRQ. There 
seems
> > to be some translation going on of a PCI IRQ number to the actual 
number
> > the system is using (due to APIC I suppose) and so the data exchange 
with
> > the device did not start.
> 
> Thanks for the extra detail.
> 
> Hmmm.  Sounds a bit weird...  Afraid I''m not sure what might be
going on
here.

What''s going on here is for example that the PCI configuration space
says
the interrupt the Ethernet card is on is IRQ ''9'', but in
reality the
system is using for example interrupt ''21'' for this device -
this happens
through translation in other parts where APIC or ACPI-related code plays a 
role. I haven''t had the time to find out what part of the system is
doing
that. Another issue is that the code in /xen/.../io_apic.c is not 
activated at all in a user/driver domain. 

I believe the architecture for access to PCI was different in Xen 1.0 
where PCI was in the HV. With 2.0 this changed and PCI access was moved to 
domain 0. Did driver domains work with that? If yes, then what was done in 
domain 0 to prepare a domain''s access to a device?

  Stefan
> 
> Cheers,
> Mark
> 
> >   Stefan
> >
> > > > >Note that giving direct physical access to a PCI device
has
security
> > > > >implications since the guest can potentially use the
cards'' DMA
> > > > > capabilities to access all of physical memory.
> > > >
> > > > Will IOMMU support help solving this security problems ?
> > >
> > > Yes but only if it enforces access permissions fully i.e. I
don''t
think
> >
> > the
> >
> > > IOEMU in AMD64 machines is sufficient.  From the looks of
Pacifica
it
> >
> > migh - 
> >
> > > have sufficient support to control the DMA problem, I''m
sure Intel
have
> >
> > a
> >
> > > similar solution (although I don''t think it''s
implemented in
Vanderpool
> >
> > -
> >
> > > they''ll probably need chipset support).
> > >
> > > Cheers,
> > > Mark
> > >
> > > > Regards,
> > > > Sting
> > > >
> > > > On 8/28/05, Mark Williamson
<mark.williamson@cl.cam.ac.uk> wrote:
> > > > > > What about other devices ? let''s say a
PCI sound card (or any
> >
> > other PCI
> >
> > > > > > device). Where is the software that should handle
it ? I
remember
> >
> > I saw
> >
> > > > > > somewhere some discussion about PCI configuration
space, but I
> >
> > don''t
> >
> > > > > > remember where.
> > > > >
> > > > > That code is in Xen itself in Xen 2.0.  Xen controls
access to
the
> >
> > PCI
> >
> > > > > configuration spaces so that guests can only see the
devices
they
> >
> > have
> >
> > > > > access to.  It also controls the IO memory / ports that
domains
are
> > > > > allowed to access in order to control PCI devices.
> > > > >
> > > > > Note that giving direct physical access to a PCI device
has
security
> > > > > implications since the guest can potentially use the
cards'' DMA
> > > > > capabilities to access all of physical memory.  The 
front/back-style
> > > > > devices do not have this limitation.
> > > > >
> > > > > Btw, I''ve laid some groundwork for a virtual
sound device but
> >
> > haven''t had
> >
> > > > > much time to hack on it yet.
> > > > >
> > > > > Cheers,
> > > > > Mark
> > >
> > > _______________________________________________
> > > Xen-devel mailing list
> > > Xen-devel@lists.xensource.com
> > > http://lists.xensource.com/xen-devel
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> > Possibly...  I would have been inclined to do it using some sort of
> > interdomain communication rather than using an emulator in Xen but
I''m
> > actually open to persuasion that I''m wrong on this point ;-)
>
> I thought about interdomain communication for emulating PCI devices. It
> ''feels'' like this would rip apart both  domain''s
PCI layers quite a bit.
Well...  I think you could hook it into the PCI layer reasonably cleanly by 
using function pointers to ensure that one set of arch functions got called 
for dom0 operation and another set get called for driver domains.

Bear in mind we don''t necessarily need to interpose the PCI
virtualisation
code on accesses to the device itself, just those to the PCI bridges.  Access 
to the device itself can be made directly (although see my comment
below).
> through translation in other parts where APIC or ACPI-related code plays a
> role. I haven''t had the time to find out what part of the system
is doing
> that. Another issue is that the code in /xen/.../io_apic.c is not
> activated at all in a user/driver domain.
That sounds like the sort of thing I was expecting but I can''t tell you
where
to look off the top of my head...
> I believe the architecture for access to PCI was different in Xen 1.0
> where PCI was in the HV. With 2.0 this changed and PCI access was moved to
> domain 0. Did driver domains work with that? If yes, then what was done in
> domain 0 to prepare a domain''s access to a device?
1.x: PCI core code and device drivers in Xen, only virtual devices in guest
2.x: PCI core code in Xen, device drivers in domains
3.x: PCI core code in dom0, no driver domains yet...

In the 1.x architecture there just weren''t any driver domains, Xen had
all the
drivers (except the graphics driver, which lived in the X server as usual).  
In 2.x the PCI core was in Xen, with guests'' PCI layer making
hypercalls
instead of doing direct hardware access.  Giving a guest a device consisted 
of: allowing it to see *just* that device in config space (reported in the 
hypercall interface), modifying its IO bitmap and memory mapping privileges 
so that it could access these things.

Since all this is yanked out of Xen now, we don''t have quite such
convenient
mechanisms for hiding things from the guest; hence config space access needs 
to be able to go through another channel, or (as you have been working 
towards) emulated somewhere.

Had you considered retasking some of the existing "IO packet" stuff as
used by
the Qemu device model to pass requests up to userspace?  Since this is only 
for device discovery the performance hit shouldn''t be an issue.  This
avoids
adding code to Xen *and* avoids special-casing in the PCI code.

While I''m on the subject, I''d personally like to see guests
granted IO access
slightly differently.  There are two ways to grant IO access on x86: change 
the IOPL (giving the guest access to all IO ports) or set IO bits in the TSS 
(giving fine grained control).  The problem with the latter is that guest 
*apps* will be able to access the hardware; essentially x86 gives you coarse 
grained control and ring-level protection, or vice-versa.

Since people often like to partition their systems using Xen, I don''t
really
like giving apps easy access to the hardware in this way.  I''d like to
have
the option of trapping IO port writes in Xen and verifying the guest''s
IO
privileges in software, then emulating the write.  It is my hope that this 
won''t hurt too much on decent hardware (e.g. devices that use an in
memory
buffer descriptor queue) and that on less clever hardware it won''t
matter too
much...

Thoughts?

Cheers,
Mark

>   Stefan
>
> > Cheers,
> > Mark
> >
> > >   Stefan
> > >
> > > > > >Note that giving direct physical access to a PCI
device has
>
> security
>
> > > > > >implications since the guest can potentially use
the cards'' DMA
> > > > > > capabilities to access all of physical memory.
> > > > >
> > > > > Will IOMMU support help solving this security problems
?
> > > >
> > > > Yes but only if it enforces access permissions fully i.e. I
don''t
>
> think
>
> > > the
> > >
> > > > IOEMU in AMD64 machines is sufficient.  From the looks of
Pacifica
>
> it
>
> > > migh -
> > >
> > > > have sufficient support to control the DMA problem,
I''m sure Intel
>
> have
>
> > > a
> > >
> > > > similar solution (although I don''t think
it''s implemented in
>
> Vanderpool
>
> > > -
> > >
> > > > they''ll probably need chipset support).
> > > >
> > > > Cheers,
> > > > Mark
> > > >
> > > > > Regards,
> > > > > Sting
> > > > >
> > > > > On 8/28/05, Mark Williamson
<mark.williamson@cl.cam.ac.uk> wrote:
> > > > > > > What about other devices ? let''s say
a PCI sound card (or any
> > >
> > > other PCI
> > >
> > > > > > > device). Where is the software that should
handle it ? I
>
> remember
>
> > > I saw
> > >
> > > > > > > somewhere some discussion about PCI
configuration space, but I
> > >
> > > don''t
> > >
> > > > > > > remember where.
> > > > > >
> > > > > > That code is in Xen itself in Xen 2.0.  Xen
controls access to
>
> the
>
> > > PCI
> > >
> > > > > > configuration spaces so that guests can only see
the devices
>
> they
>
> > > have
> > >
> > > > > > access to.  It also controls the IO memory / ports
that domains
>
> are
>
> > > > > > allowed to access in order to control PCI devices.
> > > > > >
> > > > > > Note that giving direct physical access to a PCI
device has
>
> security
>
> > > > > > implications since the guest can potentially use
the cards'' DMA
> > > > > > capabilities to access all of physical memory. 
The
>
> front/back-style
>
> > > > > > devices do not have this limitation.
> > > > > >
> > > > > > Btw, I''ve laid some groundwork for a
virtual sound device but
> > >
> > > haven''t had
> > >
> > > > > > much time to hack on it yet.
> > > > > >
> > > > > > Cheers,
> > > > > > Mark
> > > >
> > > > _______________________________________________
> > > > Xen-devel mailing list
> > > > Xen-devel@lists.xensource.com
> > > > http://lists.xensource.com/xen-devel
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Mark Williamson wrote:
> While I''m on the subject, I''d personally like to see
guests granted IO access
> slightly differently.  There are two ways to grant IO access on x86: change
> the IOPL (giving the guest access to all IO ports) or set IO bits in the
TSS
> (giving fine grained control).  The problem with the latter is that guest 
> *apps* will be able to access the hardware; essentially x86 gives you
coarse
> grained control and ring-level protection, or vice-versa.
Could XenLinux use a different TSS to run its apps?

-- 
David Hopwood <david.nospam.hopwood@blueyonder.co.uk>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

xen-devel-bounces@lists.xensource.com wrote on 09/03/2005 11:23:50 PM:
> > > Possibly...  I would have been inclined to do it using some sort
of
> > > interdomain communication rather than using an emulator in Xen
but
I''m
> > > actually open to persuasion that I''m wrong on this point
;-)
> >
> > I thought about interdomain communication for emulating PCI devices. 
It
> > ''feels'' like this would rip apart both 
domain''s PCI layers quite a
bit.
> 
> Well...  I think you could hook it into the PCI layer reasonably cleanly 
by 
> using function pointers to ensure that one set of arch functions got 
called 
> for dom0 operation and another set get called for driver domains.
All inb/w/l outb/w/l for the PCI ports 0xcf8-0xcff could also be proxied 
in domain 0 and domain 0 could make only those devices visible to the 
driver domain that are to be used by that domain. I think that doing this 
in the hypervisor through IO port or (partial) PCI bus emulation is a 
"cleaner" way, though.
> 
> Bear in mind we don''t necessarily need to interpose the PCI 
virtualisation 
> code on accesses to the device itself, just those to the PCI bridges. 
Access 
> to the device itself can be made directly (although see my comment 
below).
> > through translation in other parts where APIC or ACPI-related code 
plays a
> > role. I haven''t had the time to find out what part of the
system is
doing
> > that. Another issue is that the code in /xen/.../io_apic.c is not
> > activated at all in a user/driver domain.
> 
> That sounds like the sort of thing I was expecting but I can''t
tell you
where 
> to look off the top of my head...
Actually the IRQ translation happens through a function call like 
acpi_register_gsi() which in turn is called by pci_device_enable(). Not 
sure whether that call can work properly anywhere else than in domain 0.
> 
> > I believe the architecture for access to PCI was different in Xen 1.0
> > where PCI was in the HV. With 2.0 this changed and PCI access was 
moved to
> > domain 0. Did driver domains work with that? If yes, then what was 
done in
> > domain 0 to prepare a domain''s access to a device?
> 
> 1.x: PCI core code and device drivers in Xen, only virtual devices in 
guest
> 2.x: PCI core code in Xen, device drivers in domains
> 3.x: PCI core code in dom0, no driver domains yet...
> 
> In the 1.x architecture there just weren''t any driver domains, Xen
had
all the 
> drivers (except the graphics driver, which lived in the X server as 
usual). 
> In 2.x the PCI core was in Xen, with guests'' PCI layer making
hypercalls
> instead of doing direct hardware access.  Giving a guest a device 
consisted 
> of: allowing it to see *just* that device in config space (reported in 
the 
> hypercall interface), modifying its IO bitmap and memory mapping 
privileges 
> so that it could access these things.
> 
> Since all this is yanked out of Xen now, we don''t have quite such 
convenient 
> mechanisms for hiding things from the guest; hence config space access 
needs 
> to be able to go through another channel, or (as you have been working 
> towards) emulated somewhere.
What I have done so far is that I have presented a static config space to 
the PCI layer in user domains. This works well for reading out the config 
space, but not for writing to it. Writes that for example are coming 
through the pci_enable_device() call try to activate the device by writing 
into its config space. Now emulation is unfortunately only good as long as 
the writes are passed through to the device''s config space. Once you
start
passing writes through to the real device, you can also pass the reads 
through to it - the emulation might then only be useful for presenting an 
emulated bus entry to the PCI layer. Another requirement for properly 
reading out the config space is to have some locking done between writing 
to IO port 0xcf8 and until reading from 0xcfc. I think a proper solution 
for this would require that all domains (including domain 0) IO port 
accesses in the range of 0xcf8 to 0xcff be intercepted and either 
presented emulated parts (bus) or real devices - no domain would be 
treated differently.
> 
> Had you considered retasking some of the existing "IO packet"
stuff as
used by 
> the Qemu device model to pass requests up to userspace?  Since this is 
only 
> for device discovery the performance hit shouldn''t be an issue. 
This
avoids 
> adding code to Xen *and* avoids special-casing in the PCI code.
I had not looked at this so far.
The problem I see in any case is the IRQ number translation that is 
happening. I believe that can only be done properly in domain 0. Somewhere 
in the path of the acpi_register_gsi function call (called from 
pci_enable_device) you have to be able to calculate the IRQ number that is 
actually being used in the system. 
> 
> While I''m on the subject, I''d personally like to see
guests granted IO
access 
> slightly differently.  There are two ways to grant IO access on x86: 
change 
> the IOPL (giving the guest access to all IO ports) or set IO bits in the 
TSS 
> (giving fine grained control).  The problem with the latter is that 
guest 
> *apps* will be able to access the hardware; essentially x86 gives you 
coarse 
> grained control and ring-level protection, or vice-versa.
> 
> Since people often like to partition their systems using Xen, I
don''t
really 
> like giving apps easy access to the hardware in this way.  I''d
like to
have 
> the option of trapping IO port writes in Xen and verifying the
guest''s
IO 
> privileges in software, then emulating the write.  It is my hope that 
this 
> won''t hurt too much on decent hardware (e.g. devices that use an
in
memory 
> buffer descriptor queue) and that on less clever hardware it won''t
matter too 
> much...
> 
> Thoughts?
What I thought that could possibly be done is open up the IO port access 
of domain 0 to all ports at the beginning. A driver domain that learns 
through the PCI config space that it needs access to a range of IO ports 
could have those IO ports opened up as well (maybe automatically by the 
PCI emulation layer) and at the same time those ports could be closed in 
domain 0. Also, if the privilege bitmap in the TSS should not be done in 
HW, one could certainly do it in SW in a ''IO port emulation
layer''.  2
pages for this bitmap would be enough.

  Stefan

> 
> Cheers,
> Mark
> 
> 
> >   Stefan
> >
> > > Cheers,
> > > Mark
> > >
> > > >   Stefan
> > > >
> > > > > > >Note that giving direct physical access to a
PCI device has
> >
> > security
> >
> > > > > > >implications since the guest can potentially
use the cards''
DMA
> > > > > > > capabilities to access all of physical
memory.
> > > > > >
> > > > > > Will IOMMU support help solving this security
problems ?
> > > > >
> > > > > Yes but only if it enforces access permissions fully
i.e. I
don''t
> >
> > think
> >
> > > > the
> > > >
> > > > > IOEMU in AMD64 machines is sufficient.  From the looks
of
Pacifica
> >
> > it
> >
> > > > migh -
> > > >
> > > > > have sufficient support to control the DMA problem,
I''m sure
Intel
> >
> > have
> >
> > > > a
> > > >
> > > > > similar solution (although I don''t think
it''s implemented in
> >
> > Vanderpool
> >
> > > > -
> > > >
> > > > > they''ll probably need chipset support).
> > > > >
> > > > > Cheers,
> > > > > Mark
> > > > >
> > > > > > Regards,
> > > > > > Sting
> > > > > >
> > > > > > On 8/28/05, Mark Williamson
<mark.williamson@cl.cam.ac.uk>
wrote:
> > > > > > > > What about other devices ?
let''s say a PCI sound card (or
any
> > > >
> > > > other PCI
> > > >
> > > > > > > > device). Where is the software that
should handle it ? I
> >
> > remember
> >
> > > > I saw
> > > >
> > > > > > > > somewhere some discussion about PCI
configuration space,
but I
> > > >
> > > > don''t
> > > >
> > > > > > > > remember where.
> > > > > > >
> > > > > > > That code is in Xen itself in Xen 2.0.  Xen
controls access
to
> >
> > the
> >
> > > > PCI
> > > >
> > > > > > > configuration spaces so that guests can only
see the devices
> >
> > they
> >
> > > > have
> > > >
> > > > > > > access to.  It also controls the IO memory /
ports that
domains
> >
> > are
> >
> > > > > > > allowed to access in order to control PCI
devices.
> > > > > > >
> > > > > > > Note that giving direct physical access to a
PCI device has
> >
> > security
> >
> > > > > > > implications since the guest can potentially
use the cards''
DMA
> > > > > > > capabilities to access all of physical
memory.  The
> >
> > front/back-style
> >
> > > > > > > devices do not have this limitation.
> > > > > > >
> > > > > > > Btw, I''ve laid some groundwork for a
virtual sound device
but
> > > >
> > > > haven''t had
> > > >
> > > > > > > much time to hack on it yet.
> > > > > > >
> > > > > > > Cheers,
> > > > > > > Mark
> > > > >
> > > > > _______________________________________________
> > > > > Xen-devel mailing list
> > > > > Xen-devel@lists.xensource.com
> > > > > http://lists.xensource.com/xen-devel
> > >
> > > _______________________________________________
> > > Xen-devel mailing list
> > > Xen-devel@lists.xensource.com
> > > http://lists.xensource.com/xen-devel
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel