Xen devel - Feb 2010 - PCI passthrough w/ shared IRQ in xen/master

I cannot pass a PCI card w/ a shared IRQ to a domU.

If my PCI card is in a different slot, and gets a different IRQ, it works. 
 But in this one particular slot, it gets an IRQ that''s shared with a 
bunch of other bits of gear (USB, firewire, and ATA controller).  With the 
shared IRQ, not only can I not pass the card to my domU, my dom0 freaks 
and dies as well...

Both my dom0 and domU are using the latest xen/master pv_ops kernel and 
I''m using Xen 3.4.2 (for now).  My domU is a pv domain.

I''ve attached serial output.

Is this a bug in something (kernel or hypervisor) or a known limitation 
with pci passthrough...  I sure hope it''s a bug, cause I need to fill
all
my PCI slots with cards that are going to get passed into different 
domUs...  :-/

-Mike

---
Michael D Labriola
Electric Boat
mlabriol@gdeb.com
401-848-8871 (desk)
401-848-8513 (lab)
401-316-9844 (cell)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 19/02/2010 15:56, "Michael D Labriola" <mlabriol@gdeb.com>
wrote:
> Is this a bug in something (kernel or hypervisor) or a known limitation
> with pci passthrough...  I sure hope it''s a bug, cause I need to
fill all
> my PCI slots with cards that are going to get passed into different
> domUs...  :-/
Looks like dom0 doesn''t like receiving interrupts that are actually
intended
for the pass-thru device in the domU, complains and shuts down the interrupt
line, and then gets upset that its SATA IRQ has disappeared.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

xen-devel-bounces@lists.xensource.com wrote on 02/19/2010 11:06:51 AM:
> On 19/02/2010 15:56, "Michael D Labriola"
<mlabriol@gdeb.com> wrote:
> 
> > Is this a bug in something (kernel or hypervisor) or a known 
limitation
> > with pci passthrough...  I sure hope it''s a bug, cause I need
to fill
all
> > my PCI slots with cards that are going to get passed into different
> > domUs...  :-/
> 
> Looks like dom0 doesn''t like receiving interrupts that are
actually
intended
> for the pass-thru device in the domU, complains and shuts down the 
interrupt
> line, and then gets upset that its SATA IRQ has disappeared.
Yeah, that''s how I interpreted that as well.  I''m having fun
running fsck
over here now.

Does this point towards a bug in xen/master''s dom0 IRQ handling?

-Mike

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> Is this a bug in something (kernel or hypervisor) or a known limitation
> with pci passthrough...  I sure hope it''s a bug, cause I need to
fill all
> my PCI slots with cards that are going to get passed into different
> domUs...  :-/
I''m also very curious if this will be resolved at some point, so far
I''ve
run into similar problems using both 3.4-testing as well as 4.0.

When you do passthrough the device which has a shared IRQ, the domU will
load the appropriate driver, but Xen refuses to assign the IRQ to the
domU.
In your case (and mine as well), the driver in domU is not aware that it
has no IRQ, and thus starts generating interupts which are then caught by
Dom0 as spurious interrupts, in my case even causing Dom0 to crash at
times.

My way around this, is to try and move the other device IRQs out of the
way (the ones that matter seem to be MSI capable), but i understand
that''s
not always possible.
If your other device is PCI based, and has no MSI, then for now I''m
affraid you''re out of luck...

Like you, I do hope this is considered a bug that will be fixed in a later
release... anyone know what the xen policy is on this?


Mark.




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 19/02/2010 16:15, "Mark Hurenkamp" <mark.hurenkamp@xs4all.nl>
wrote:
> When you do passthrough the device which has a shared IRQ, the domU will
> load the appropriate driver, but Xen refuses to assign the IRQ to the
> domU.
> In your case (and mine as well), the driver in domU is not aware that it
> has no IRQ, and thus starts generating interupts which are then caught by
> Dom0 as spurious interrupts, in my case even causing Dom0 to crash at
> times.
Are you sure the domU doesn''t get the interrupts as well? Correct
behaviour
in Xen will be to send the interrupt to both dom0 and domU - since Xen
can''t
work out which of the two it properly belongs to (and it could be both).

I would suggest try commenting out the content of the function
kernel/irq/spurious.c:note_interrupt(). That will stop dom0 from reacting to
the ''spurious'' irqs that it sees and ignore them instead. See
if that works
for you.

 -- Keir
> My way around this, is to try and move the other device IRQs out of the
> way (the ones that matter seem to be MSI capable), but i understand
that''s
> not always possible.
> If your other device is PCI based, and has no MSI, then for now
I''m
> affraid you''re out of luck...
> 
> Like you, I do hope this is considered a bug that will be fixed in a later
> release... anyone know what the xen policy is on this?


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

xen-devel-bounces@lists.xensource.com wrote on 02/19/2010 11:15:21 AM:
> > Is this a bug in something (kernel or hypervisor) or a known 
limitation
> > with pci passthrough...  I sure hope it''s a bug, cause I need
to fill
all
> > my PCI slots with cards that are going to get passed into different
> > domUs...  :-/
> 
> I''m also very curious if this will be resolved at some point, so
far
I''ve
> run into similar problems using both 3.4-testing as well as 4.0.
> 
> When you do passthrough the device which has a shared IRQ, the domU will
> load the appropriate driver, but Xen refuses to assign the IRQ to the
> domU.
> In your case (and mine as well), the driver in domU is not aware that it
> has no IRQ, and thus starts generating interupts which are then caught 
by
> Dom0 as spurious interrupts, in my case even causing Dom0 to crash at
> times.
This actually sounds like 2 bugs when you describe it like that:

1. Can''t assign shared IRQ PCI device to domU

2. Dom0 commit suicide when it gets spurious interrupts from a shared IRQ.
> 
> My way around this, is to try and move the other device IRQs out of the
> way (the ones that matter seem to be MSI capable), but i understand 
that''s
> not always possible.
> If your other device is PCI based, and has no MSI, then for now
I''m
> affraid you''re out of luck...
The IRQ in question is shared with these devices (along with my NIC):

USB Controller
IDE Interface
FireWire

I don''t really know what MSI is...  I''m assuming I''m
out of luck, though.
> 
> Like you, I do hope this is considered a bug that will be fixed in a 
later
> release... anyone know what the xen policy is on this?
Well, if somebody wants to help with the kernel code, I can test it.

-Mike

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser <keir.fraser@eu.citrix.com> wrote on 02/19/2010 11:24:28 AM:
> On 19/02/2010 16:15, "Mark Hurenkamp"
<mark.hurenkamp@xs4all.nl> wrote:
> 
> > When you do passthrough the device which has a shared IRQ, the domU 
will
> > load the appropriate driver, but Xen refuses to assign the IRQ to the
> > domU.
> > In your case (and mine as well), the driver in domU is not aware that 
it
> > has no IRQ, and thus starts generating interupts which are then caught
by
> > Dom0 as spurious interrupts, in my case even causing Dom0 to crash at
> > times.
> 
> Are you sure the domU doesn''t get the interrupts as well? Correct 
behaviour
> in Xen will be to send the interrupt to both dom0 and domU - since Xen 
can''t
> work out which of the two it properly belongs to (and it could be both).
> 
> I would suggest try commenting out the content of the function
> kernel/irq/spurious.c:note_interrupt(). That will stop dom0 from 
reacting to
> the ''spurious'' irqs that it sees and ignore them instead.
See if that
works
> for you.
On it.  Let you know what happens.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Am 19.02.2010 17:06, schrieb Keir Fraser:
> On 19/02/2010 15:56, "Michael D Labriola"
<mlabriol@gdeb.com> wrote:
> 
Hello,
>> Is this a bug in something (kernel or hypervisor) or a known limitation
>> with pci passthrough...  I sure hope it''s a bug, cause I need
to fill all
>> my PCI slots with cards that are going to get passed into different
>> domUs...  :-/
> 
> Looks like dom0 doesn''t like receiving interrupts that are
actually intended
> for the pass-thru device in the domU, complains and shuts down the
interrupt
> line, and then gets upset that its SATA IRQ has disappeared.
> 
sounds like my Problem with my new small Hardware.
When i passthrough a PCI device IRQ 16 is disabled and then SATA goes down.

Regards,
Stefan Kuhne



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

xen-devel-bounces@lists.xensource.com wrote on 02/19/2010 11:26:11 AM:
> Keir Fraser <keir.fraser@eu.citrix.com> wrote on 02/19/2010 11:24:28
AM:
> 
> > On 19/02/2010 16:15, "Mark Hurenkamp"
<mark.hurenkamp@xs4all.nl>
wrote:
> > 
> > > When you do passthrough the device which has a shared IRQ, the
domU
> will
> > > load the appropriate driver, but Xen refuses to assign the IRQ to
the
> > > domU.
> > > In your case (and mine as well), the driver in domU is not aware 
that 
> it
> > > has no IRQ, and thus starts generating interupts which are then 
caught 
> by
> > > Dom0 as spurious interrupts, in my case even causing Dom0 to
crash
at
> > > times.
> > 
> > Are you sure the domU doesn''t get the interrupts as well?
Correct
> behaviour
> > in Xen will be to send the interrupt to both dom0 and domU - since Xen
> can''t
> > work out which of the two it properly belongs to (and it could be 
both).
> > 
> > I would suggest try commenting out the content of the function
> > kernel/irq/spurious.c:note_interrupt(). That will stop dom0 from 
> reacting to
> > the ''spurious'' irqs that it sees and ignore them
instead. See if that
> works
> > for you.
> 
> On it.  Let you know what happens.
Commented out the entire contents of note_interrupt().  Only installed the 
new kernel on my dom0.  That does in fact keep my dom0 from killing my 
SATA controller, but my PCI ATM card still doesn''t quite work in the
domU.
 How exactly do I tell if the domU is getting the interrupts?

The card works well enough for the ATM daemons to set up a PVC, but when I 
try to ping the host on the other side I get nothing.  Actually, after a 
few minutes, I start getting this message:

ping: sendmsg: No buffer space available

-Mike

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Feb 19, 2010 at 10:56:08AM -0500, Michael D Labriola
wrote:
> I cannot pass a PCI card w/ a shared IRQ to a domU.
> 
> If my PCI card is in a different slot, and gets a different IRQ, it works. 
>  But in this one particular slot, it gets an IRQ that''s shared
with a
> bunch of other bits of gear (USB, firewire, and ATA controller).  With the 
> shared IRQ, not only can I not pass the card to my domU, my dom0 freaks 
> and dies as well...
> 
> Both my dom0 and domU are using the latest xen/master pv_ops kernel and 
> I''m using Xen 3.4.2 (for now).  My domU is a pv domain.
There is a missing patch that I haven''t pushed in the pcifront branch:

diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
index e138053..923de2e 100644
--- a/arch/x86/pci/xen.c
+++ b/arch/x86/pci/xen.c
@@ -25,7 +25,7 @@ static int xen_pcifront_enable_irq(struct pci_dev *dev)
 	if (dev->irq < 0)
 		return -EINVAL;
 
-	rc = xen_allocate_pirq(dev->irq, 0, "pcifront");
+	rc = xen_allocate_pirq(dev->irq, 1 /* share */, "pcifront");
 	if (rc < 0) {
 		dev_warn(&dev->dev, "Xen PCI IRQ: %d, failed to
register:%d\n",
 			 dev->irq, rc);


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 19/02/2010 16:55, "Michael D Labriola" <mlabriol@gdeb.com>
wrote:
> Commented out the entire contents of note_interrupt().  Only installed the
> new kernel on my dom0.  That does in fact keep my dom0 from killing my
> SATA controller, but my PCI ATM card still doesn''t quite work in
the domU.
>  How exactly do I tell if the domU is getting the interrupts?
cat /proc/interrupts

You can also use Xen''s ''i'' debug key to dump its
guest interrupt
information, and see where it thinks it is routing each physical interrupt
line to.
> The card works well enough for the ATM daemons to set up a PVC, but when I
> try to ping the host on the other side I get nothing.  Actually, after a
> few minutes, I start getting this message:
> 
> ping: sendmsg: No buffer space available
Hm, well, failure to transmit could be various things other than simply not
receiving interrupts.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

xen-devel-bounces@lists.xensource.com wrote on 02/19/2010 11:55:27 AM:
> xen-devel-bounces@lists.xensource.com wrote on 02/19/2010 11:26:11 AM:
> 
> > Keir Fraser <keir.fraser@eu.citrix.com> wrote on 02/19/2010
11:24:28
AM:
> > 
> > > On 19/02/2010 16:15, "Mark Hurenkamp"
<mark.hurenkamp@xs4all.nl>
> wrote:
> > > 
> > > > When you do passthrough the device which has a shared IRQ,
the
domU 
> > will
> > > > load the appropriate driver, but Xen refuses to assign the
IRQ to
> the
> > > > domU.
> > > > In your case (and mine as well), the driver in domU is not
aware
> that 
> > it
> > > > has no IRQ, and thus starts generating interupts which are
then
> caught 
> > by
> > > > Dom0 as spurious interrupts, in my case even causing Dom0 to
crash
> at
> > > > times.
> > > 
> > > Are you sure the domU doesn''t get the interrupts as
well? Correct
> > behaviour
> > > in Xen will be to send the interrupt to both dom0 and domU -
since
Xen 
> 
> > can''t
> > > work out which of the two it properly belongs to (and it could be
> both).
> > > 
> > > I would suggest try commenting out the content of the function
> > > kernel/irq/spurious.c:note_interrupt(). That will stop dom0 from 
> > reacting to
> > > the ''spurious'' irqs that it sees and ignore
them instead. See if
that 
> > works
> > > for you.
> > 
> > On it.  Let you know what happens.
> 
> Commented out the entire contents of note_interrupt().  Only installed 
the 
> new kernel on my dom0.  That does in fact keep my dom0 from killing my 
> SATA controller, but my PCI ATM card still doesn''t quite work in
the
domU. 
>  How exactly do I tell if the domU is getting the interrupts?
> 
> The card works well enough for the ATM daemons to set up a PVC, but when 
I 
> try to ping the host on the other side I get nothing.  Actually, after a 
> few minutes, I start getting this message:
> 
> ping: sendmsg: No buffer space available
Ok, I''ve attached the serial output from my dom0 w/out any 
note_interrupt() content, the output of dmesg on my bad domU, and the 
contents of /proc/interrupts on my bad domU.

I also included dmesg and interrupts from a "good" domU.  This other
domU
is running on the same dom0 with the same model PCI ATM card (HE155) in a 
different PCI slot with a unique IRQ.

I''m gonna go verify that the ATM w/ shared IRQ works in dom0...

-Mike











_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

xen-devel-bounces@lists.xensource.com wrote on 02/19/2010 12:07:08 PM:
> On Fri, Feb 19, 2010 at 10:56:08AM -0500, Michael D Labriola wrote:
> > I cannot pass a PCI card w/ a shared IRQ to a domU.
> > 
> > If my PCI card is in a different slot, and gets a different IRQ, it 
works. 
> >  But in this one particular slot, it gets an IRQ that''s
shared with a
> > bunch of other bits of gear (USB, firewire, and ATA controller).  With
the 
> > shared IRQ, not only can I not pass the card to my domU, my dom0 
freaks 
> > and dies as well...
> > 
> > Both my dom0 and domU are using the latest xen/master pv_ops kernel 
and 
> > I''m using Xen 3.4.2 (for now).  My domU is a pv domain.
> 
> There is a missing patch that I haven''t pushed in the pcifront
branch:
> 
> diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
> index e138053..923de2e 100644
> --- a/arch/x86/pci/xen.c
> +++ b/arch/x86/pci/xen.c
> @@ -25,7 +25,7 @@ static int xen_pcifront_enable_irq(struct pci_dev 
*dev)
>     if (dev->irq < 0)
>        return -EINVAL;
> 
> -   rc = xen_allocate_pirq(dev->irq, 0, "pcifront");
> +   rc = xen_allocate_pirq(dev->irq, 1 /* share */,
"pcifront");
>     if (rc < 0) {
>        dev_warn(&dev->dev, "Xen PCI IRQ: %d, failed to
register:%d\n",
>            dev->irq, rc);
> 
> 

HA!  That''s an important one-line diff!  Now my ATM card works w/
shared
IRQ on domU.  :-)

Now we just have to get a better fix for dom0 than commenting out the 
entire note_interrupt() function...

-Mike


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Am 19.02.2010 18:49, schrieb Michael D Labriola:
> HA!  That''s an important one-line diff!  Now my ATM card works w/
shared
> IRQ on domU.  :-)
> 
> Now we just have to get a better fix for dom0 than commenting out the 
> entire note_interrupt() function...
> 
Is this an Kernel or Xen Patch?
I''d try this with xen-4.0.0 an 2.6.31.6 pv_ops Kernel.

Regards,
Stefan Kuhne



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Am 19.02.2010 19:27, schrieb Stefan Kuhne:
> Am 19.02.2010 18:49, schrieb Michael D Labriola:
> 
>> HA!  That''s an important one-line diff!  Now my ATM card works
w/ shared
>> IRQ on domU.  :-)
>>
>> Now we just have to get a better fix for dom0 than commenting out the 
>> entire note_interrupt() function...
>>
> Is this an Kernel or Xen Patch?
It is a Kernel Patch.
> I''d try this with xen-4.0.0 an 2.6.31.6 pv_ops Kernel.
> 
I seams to work for me.
DomU boots without total crash.

Regards,
Stefan Kuhne



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

xen-devel-bounces@lists.xensource.com wrote on 02/19/2010 12:49:40 PM:
> xen-devel-bounces@lists.xensource.com wrote on 02/19/2010 12:07:08 PM:
> 
> > On Fri, Feb 19, 2010 at 10:56:08AM -0500, Michael D Labriola wrote:
> > > I cannot pass a PCI card w/ a shared IRQ to a domU.
> > > 
> > > If my PCI card is in a different slot, and gets a different IRQ,
it
> works. 
> > >  But in this one particular slot, it gets an IRQ that''s
shared with
a 
> > > bunch of other bits of gear (USB, firewire, and ATA controller). 
With 
> the 
> > > shared IRQ, not only can I not pass the card to my domU, my dom0 
> freaks 
> > > and dies as well...
> > > 
> > > Both my dom0 and domU are using the latest xen/master pv_ops
kernel
> and 
> > > I''m using Xen 3.4.2 (for now).  My domU is a pv domain.
> > 
> > There is a missing patch that I haven''t pushed in the
pcifront branch:
> > 
> > diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
> > index e138053..923de2e 100644
> > --- a/arch/x86/pci/xen.c
> > +++ b/arch/x86/pci/xen.c
> > @@ -25,7 +25,7 @@ static int xen_pcifront_enable_irq(struct pci_dev 
> *dev)
> >     if (dev->irq < 0)
> >        return -EINVAL;
> > 
> > -   rc = xen_allocate_pirq(dev->irq, 0, "pcifront");
> > +   rc = xen_allocate_pirq(dev->irq, 1 /* share */,
"pcifront");
> >     if (rc < 0) {
> >        dev_warn(&dev->dev, "Xen PCI IRQ: %d, failed to
register:%d\n",
> >            dev->irq, rc);
> > 
> > 
> 
> 
> HA!  That''s an important one-line diff!  Now my ATM card works w/
shared
> IRQ on domU.  :-)
> 
> Now we just have to get a better fix for dom0 than commenting out the 
> entire note_interrupt() function...
Actually, I just verified that that patch makes the note_interrupt() 
gutting unnecessary.  With just that one line changed, I''m up and
running
perfectly.

-Mike


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 19/02/2010 19:14, "Michael D Labriola" <mlabriol@gdeb.com>
wrote:
>> IRQ on domU.  :-)
>> 
>> Now we just have to get a better fix for dom0 than commenting out the
>> entire note_interrupt() function...
> 
> Actually, I just verified that that patch makes the note_interrupt()
> gutting unnecessary.  With just that one line changed, I''m up and
running
> perfectly.
Yeah, that probably means that pv_ops already has logic to disable that
spurious interrupt logic when an interrupt is shared between multiple
guests. If your domU was failing to bind to the interrupt line, that would
mean the interrupt would not appear to be shared (from dom0 kernel pov) and
then when the domU-passed-thru device starts generating interrupts dom0
acted on them as really spurious (since it thinks it owns all devices that
might generate that interrupt). So the note_interrupt() hack treated that
symptom but not the actual original cause of the troubles.

Interestingly, that does mean this is a way an errant domU can really spoil
dom0''s day....

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> > > There is a missing patch that I haven''t pushed in the
pcifront branch:
> > > 
> > > diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
> > > index e138053..923de2e 100644
> > > --- a/arch/x86/pci/xen.c
> > > +++ b/arch/x86/pci/xen.c
> > > @@ -25,7 +25,7 @@ static int xen_pcifront_enable_irq(struct
pci_dev
> > *dev)
> > >     if (dev->irq < 0)
> > >        return -EINVAL;
> > > 
> > > -   rc = xen_allocate_pirq(dev->irq, 0, "pcifront");
> > > +   rc = xen_allocate_pirq(dev->irq, 1 /* share */,
"pcifront");
> > >     if (rc < 0) {
> > >        dev_warn(&dev->dev, "Xen PCI IRQ: %d, failed
to register:%d\n",
> > >            dev->irq, rc);
> > > 
> > > 
> > 
> > 
> > HA!  That''s an important one-line diff!  Now my ATM card
works w/ shared
> 
> > IRQ on domU.  :-)
> > 
> > Now we just have to get a better fix for dom0 than commenting out the 
> > entire note_interrupt() function...
> 
> Actually, I just verified that that patch makes the note_interrupt() 
> gutting unnecessary.  With just that one line changed, I''m up and
running
> perfectly.
Excellent.

Jeremy, could you pull
git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git pcifront-2.6.31.6 

in your xen/pcifront and merge it to xen/master, please?


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 02/19/2010 01:45 PM, Konrad Rzeszutek Wilk wrote:
> Excellent.
>
> Jeremy, could you pull
> git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git
pcifront-2.6.31.6
>
> in your xen/pcifront and merge it to xen/master, please?
>    
Done.

     J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel