HI,
We are currently trying to pass through the USB controller (OHCI)
using 1-to-1 mapping domain (mainly based on the Neo-1-to-1 patch) on
a Non-VT-d machine. If we do not plug in any devices, the HVM works
fine, but as long as we plug in one USB device, the HVM domain
crashes.
From the serial output we find that interrupts keep firing at the
IRQ line that was passed through. The Neo-1-to-1 patch uses a so
called CPA (change polarity algorithm) to handle interrupt pass
through, we are not very clear about this idea and even why it should
be done. Should any body explain it to us, that will be very helpful.
We are also trying to pass through an IEEE 1394 OHCI Controller,
under FC6 HVM domain, it keeps on firing interrupts after the boot
process goes to udev step, which crashes the system. It is even worse
under WinXP, it crashes the system and makes permanent damage to the
QEMU disk image, we have know idea why this happens.
By the way, we are using an AMD processor with SVM support.
Thank you.
Cheers
ConcreteChen@gmail.com
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Hi, a. The OHCI USB Controller has 4K MMIO region and it is aligned in DOM0. b. Yes, but 1) the device that sharing this IRQ line never generate any interrupts (it is the HDA device) 2) when we bind the interrupt, we free any sharing IRQs explicitly in Xen by calling free_irq We even 1-to-1 mapped the MMIO region. The Neo patch adds a call to pt_intr_assist() between pt_update_irq() and hvm_set_callback_irq_level() in vmx_intr_assist(), since we are using SVM, we add the call to pt_intr_assist() between pt_update_irq() and hvm_set_callback_irq_level() in svm_intr_assist(). // We have extracted the changes in the Neo patch and appied it to the standard Xen-3.1 version Thank you. 2008/3/20, Alex Novik <alex@neocleus.com>:> Hi, > I''m one of the guys who is responsible for this patch. So, let''s try > to understand what''s happening there... > 1. I''d like you to check following things for me: > a. Is your USB controller has mmio space aligned? Look at the lspci > in DOM0 and verify that all MMIO spaces are aligned to 4kb. If not, fix > it manually (with setpci), if it helps look for mmio alignment patch for > DOM0. > b. Do you have any devices, beside USB on the same IRQ line? "cat > /proc/interrupts/ " will show you this. If you do, it comes to another > problem, which we called interdomain sharing. Look for the discussion > related to that issue and apply patch. To verify that this is the > problem, just remove drivers of all the devices that are using the same > IRQ () > I''m pretty positive that your interrupt storming related to one of this > two issues. > Anyway, keep me informed. > Best regards, > Alex. > > > > > -----Original Message----- > From: xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of ?? > Sent: Thursday, March 20, 2008 10:50 AM > To: xen-devel > Subject: [Xen-devel] About passing through USB devices in HVM domain > > HI, > We are currently trying to pass through the USB controller (OHCI) > using 1-to-1 mapping domain (mainly based on the Neo-1-to-1 patch) on > a Non-VT-d machine. If we do not plug in any devices, the HVM works > fine, but as long as we plug in one USB device, the HVM domain > crashes. > From the serial output we find that interrupts keep firing at the > IRQ line that was passed through. The Neo-1-to-1 patch uses a so > called CPA (change polarity algorithm) to handle interrupt pass > through, we are not very clear about this idea and even why it should > be done. Should any body explain it to us, that will be very helpful. > We are also trying to pass through an IEEE 1394 OHCI Controller, > under FC6 HVM domain, it keeps on firing interrupts after the boot > process goes to udev step, which crashes the system. It is even worse > under WinXP, it crashes the system and makes permanent damage to the > QEMU disk image, we have know idea why this happens. > By the way, we are using an AMD processor with SVM support. > Thank you. > > Cheers > > ConcreteChen@gmail.com > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
1. How does DOM0 get the interrupt of the pt device? According to irq.c, do_IRQ_pt handles the pass through interrupt and __do_IRQ_guest will never be executed in the pass through IRQ case. Maybe we have misunderstand the code here, please point out:) 2. Just in pt_irq_create_bind_neo, after finding desc->action != NULL, we do not goto out but explicitly call free_irq() to free the irq. OK, we will try remove the driver. But the IEEE 1394 controller owns its IRQ line and does not share it with any other device, but it still results in interrupt storming when passed through. 3. Hopfully this is not a problem of SVM. The interrupt is actually injected into the HVM domain, we even have the dump information when we pluged in the USB device under FC6 HVM domain, it seems to follow the way: 4. We have added the do_IRQ_pt() call in do_IRQ(). And could you help explaining the whole CPA stuff, especially why it is needed:) Thank you. 2008/3/20, Alex Novik <alex@neocleus.com>:> A.ok. > B. 1.It doesn't matter if it is generating interrupt itself or not. The problem is? as soon as pt device generates interrupt it will go to DOM0, which is unable to handle this interrupt. Results in interrupt storming. > 2. Do you do this before or after binding? Anyway, the simplest way to do it, is just rmmod the driver from dom0 before running hvm. Try it pls. Just to make sure.. > 3. I'm not positive about svm. We didn't try it. But your problem sounds like if interrupt is not reaching hvm or hvm is not able to access the device to clear the interrupt. Try to verify that interrupt was sent to hvm (prints or whatever you're using). The good point to start is inside pt_intr_assist. It does the injection, verify that this is happeneing... > 4. Hopefully, you didn't forget to make changes to irq.c as well, do you? > > -----Original Message----- > From: 陈诚 [mailto:concretechen@gmail.com] > Sent: Thursday, March 20, 2008 1:53 PM > To: Alex Novik; xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] About passing through USB devices in HVM domain > > Hi, > a. The OHCI USB Controller has 4K MMIO region and it is aligned in DOM0. > b. Yes, but 1) the device that sharing this IRQ line never generate any interrupts (it is the HDA device) 2) when we bind the interrupt, we free any sharing IRQs explicitly in Xen by calling free_irq > > We even 1-to-1 mapped the MMIO region. > > The Neo patch adds a call to pt_intr_assist() between pt_update_irq() and hvm_set_callback_irq_level() in vmx_intr_assist(), since we are using SVM, we add the call to pt_intr_assist() between pt_update_irq() and hvm_set_callback_irq_level() in svm_intr_assist(). > // We have extracted the changes in the Neo patch and appied it to the standard Xen-3.1 version > > Thank you. > > > > 2008/3/20, Alex Novik <alex@neocleus.com>: > > Hi, > > I'm one of the guys who is responsible for this patch. So, let's try > > to understand what's happening there... > > 1. I'd like you to check following things for me: > > a. Is your USB controller has mmio space aligned? Look at the > > lspci in DOM0 and verify that all MMIO spaces are aligned to 4kb. If > > not, fix it manually (with setpci), if it helps look for mmio > > alignment patch for DOM0. > > b. Do you have any devices, beside USB on the same IRQ line? "cat > > /proc/interrupts/ " will show you this. If you do, it comes to another > > problem, which we called interdomain sharing. Look for the discussion > > related to that issue and apply patch. To verify that this is the > > problem, just remove drivers of all the devices that are using the > > same IRQ () I'm pretty positive that your interrupt storming related > > to one of this two issues. > > Anyway, keep me informed. > > Best regards, > > Alex. > > > > > > > > > > -----Original Message----- > > From: xen-devel-bounces@lists.xensource.com > > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of ?? > > Sent: Thursday, March 20, 2008 10:50 AM > > To: xen-devel > > Subject: [Xen-devel] About passing through USB devices in HVM domain > > > > HI, > > We are currently trying to pass through the USB controller (OHCI) > > using 1-to-1 mapping domain (mainly based on the Neo-1-to-1 patch) on > > a Non-VT-d machine. If we do not plug in any devices, the HVM works > > fine, but as long as we plug in one USB device, the HVM domain > > crashes. > > From the serial output we find that interrupts keep firing at the > > IRQ line that was passed through. The Neo-1-to-1 patch uses a so > > called CPA (change polarity algorithm) to handle interrupt pass > > through, we are not very clear about this idea and even why it should > > be done. Should any body explain it to us, that will be very helpful. > > We are also trying to pass through an IEEE 1394 OHCI Controller, > > under FC6 HVM domain, it keeps on firing interrupts after the boot > > process goes to udev step, which crashes the system. It is even worse > > under WinXP, it crashes the system and makes permanent damage to the > > QEMU disk image, we have know idea why this happens. > > By the way, we are using an AMD processor with SVM support. > > Thank you. > > > > Cheers > > > > ConcreteChen@gmail.com > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Sorry, at point 3 I missed the the dump information. It seems to follow the way: do_IRQ() __do_IRQ() note_interrupt() __report_bad_irq That is more than 99900 interrupts out of 100000 are unhandled. 2008/3/20, Alex Novik <alex@neocleus.com>:> A.ok. > B. 1.It doesn't matter if it is generating interrupt itself or not. The problem is? as soon as pt device generates interrupt it will go to DOM0, which is unable to handle this interrupt. Results in interrupt storming. > 2. Do you do this before or after binding? Anyway, the simplest way to do it, is just rmmod the driver from dom0 before running hvm. Try it pls. Just to make sure.. > 3. I'm not positive about svm. We didn't try it. But your problem sounds like if interrupt is not reaching hvm or hvm is not able to access the device to clear the interrupt. Try to verify that interrupt was sent to hvm (prints or whatever you're using). The good point to start is inside pt_intr_assist. It does the injection, verify that this is happeneing... > 4. Hopefully, you didn't forget to make changes to irq.c as well, do you? > > -----Original Message----- > From: 陈诚 [mailto:concretechen@gmail.com] > Sent: Thursday, March 20, 2008 1:53 PM > To: Alex Novik; xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] About passing through USB devices in HVM domain > > Hi, > a. The OHCI USB Controller has 4K MMIO region and it is aligned in DOM0. > b. Yes, but 1) the device that sharing this IRQ line never generate any interrupts (it is the HDA device) 2) when we bind the interrupt, we free any sharing IRQs explicitly in Xen by calling free_irq > > We even 1-to-1 mapped the MMIO region. > > The Neo patch adds a call to pt_intr_assist() between pt_update_irq() and hvm_set_callback_irq_level() in vmx_intr_assist(), since we are using SVM, we add the call to pt_intr_assist() between pt_update_irq() and hvm_set_callback_irq_level() in svm_intr_assist(). > // We have extracted the changes in the Neo patch and appied it to the standard Xen-3.1 version > > Thank you. > > > > 2008/3/20, Alex Novik <alex@neocleus.com>: > > Hi, > > I'm one of the guys who is responsible for this patch. So, let's try > > to understand what's happening there... > > 1. I'd like you to check following things for me: > > a. Is your USB controller has mmio space aligned? Look at the > > lspci in DOM0 and verify that all MMIO spaces are aligned to 4kb. If > > not, fix it manually (with setpci), if it helps look for mmio > > alignment patch for DOM0. > > b. Do you have any devices, beside USB on the same IRQ line? "cat > > /proc/interrupts/ " will show you this. If you do, it comes to another > > problem, which we called interdomain sharing. Look for the discussion > > related to that issue and apply patch. To verify that this is the > > problem, just remove drivers of all the devices that are using the > > same IRQ () I'm pretty positive that your interrupt storming related > > to one of this two issues. > > Anyway, keep me informed. > > Best regards, > > Alex. > > > > > > > > > > -----Original Message----- > > From: xen-devel-bounces@lists.xensource.com > > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of ?? > > Sent: Thursday, March 20, 2008 10:50 AM > > To: xen-devel > > Subject: [Xen-devel] About passing through USB devices in HVM domain > > > > HI, > > We are currently trying to pass through the USB controller (OHCI) > > using 1-to-1 mapping domain (mainly based on the Neo-1-to-1 patch) on > > a Non-VT-d machine. If we do not plug in any devices, the HVM works > > fine, but as long as we plug in one USB device, the HVM domain > > crashes. > > From the serial output we find that interrupts keep firing at the > > IRQ line that was passed through. The Neo-1-to-1 patch uses a so > > called CPA (change polarity algorithm) to handle interrupt pass > > through, we are not very clear about this idea and even why it should > > be done. Should any body explain it to us, that will be very helpful. > > We are also trying to pass through an IEEE 1394 OHCI Controller, > > under FC6 HVM domain, it keeps on firing interrupts after the boot > > process goes to udev step, which crashes the system. It is even worse > > under WinXP, it crashes the system and makes permanent damage to the > > QEMU disk image, we have know idea why this happens. > > By the way, we are using an AMD processor with SVM support. > > Thank you. > > > > Cheers > > > > ConcreteChen@gmail.com > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel