Tom Rotenberg
2009-Nov-24 18:42 UTC
[Xen-devel] Bug when disabling/enabling a PT device with MSI enabled
Hi, I am using the xen-3,4, and i am doing pass-through for a device with MSI enabled (the device of-course supports MSI) to a domU with Windows XP. When i disable and then re-enable the device using Windows device manager, everything looks like it''s working fine, but the device seems to malfunction. After digging a little bit, i saw that the problem is, because the bit of the interrupt-status (bit 3) in the status register (offset 0x6 in the PCI config space) was turned on somehow - this proibably caused the MSI to malfunction (because the IntX assertion was enabled - thus disabling the MSI) - and this lead to the problem i experienced. Any ideas on how to solve this issue? Tom _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2009-Nov-24 19:48 UTC
Re: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled
On Tue, Nov 24, 2009 at 08:42:49PM +0200, Tom Rotenberg wrote:> Hi, > > I am using the xen-3,4, and i am doing pass-through for a device with > MSI enabled (the device of-course supports MSI) to a domU with Windows > XP. When i disable and then re-enable the device using Windows device > manager, everything looks like it''s working fine, but the device seems > to malfunction. > After digging a little bit, i saw that the problem is, because the bit > of the interrupt-status (bit 3) in the status register (offset 0x6 in > the PCI config space) was turned on somehow - this proibably caused > the MSI to malfunction (because the IntX assertion was enabled - thus > disabling the MSI) - and this lead to the problem i experienced. > > Any ideas on how to solve this issue?Looks like QEMU is not doing its job. It ought to be tracking when an MSI devices transforms to a INTx PCI device (and vice-versa) and make the appropiate Xen calls. Look in ioemu-dir/hw/pass-through.c _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tom Rotenberg
2009-Nov-24 20:27 UTC
Re: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled
Can u please supply more details? how can i fix this issue? I think that when the hardware reset occured, the MSI was disabled, and that needs to be re-enabled again... no? On Tue, Nov 24, 2009 at 9:48 PM, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:> On Tue, Nov 24, 2009 at 08:42:49PM +0200, Tom Rotenberg wrote: >> Hi, >> >> I am using the xen-3,4, and i am doing pass-through for a device with >> MSI enabled (the device of-course supports MSI) to a domU with Windows >> XP. When i disable and then re-enable the device using Windows device >> manager, everything looks like it''s working fine, but the device seems >> to malfunction. >> After digging a little bit, i saw that the problem is, because the bit >> of the interrupt-status (bit 3) in the status register (offset 0x6 in >> the PCI config space) was turned on somehow - this proibably caused >> the MSI to malfunction (because the IntX assertion was enabled - thus >> disabling the MSI) - and this lead to the problem i experienced. >> >> Any ideas on how to solve this issue? > > Looks like QEMU is not doing its job. It ought to be tracking > when an MSI devices transforms to a INTx PCI device (and vice-versa) and > make the appropiate Xen calls. Look in ioemu-dir/hw/pass-through.c >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2009-Nov-24 20:46 UTC
Re: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled
On Tue, Nov 24, 2009 at 10:27:38PM +0200, Tom Rotenberg wrote:> Can u please supply more details? how can i fix this issue?.. snip ..> > make the appropiate Xen calls. Look in ioemu-dir/hw/pass-through.cStart digging in the code. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Kamala Narasimhan
2009-Nov-25 02:25 UTC
RE: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled
Couple of questions - 1) What kind of device are you referring to below? Display by any chance? 2) Which tree is your dom0 kernel based on? There is a chance you are missing some msi patches in your dom0 kernel. You might want to do a quick search for msi patches in http://xenbits.xen.org/staging/linux-2.6.18-xen.hg?rev=msi and see if you are missing anything obvious. I recently encountered a similar issue in a slightly different situation, a subset of the problem was caused because of missing msi patches. Kamala> -----Original Message----- > From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel- > bounces@lists.xensource.com] On Behalf Of Tom Rotenberg > Sent: Tuesday, November 24, 2009 1:43 PM > To: xen-devel@lists.xensource.com > Subject: [Xen-devel] Bug when disabling/enabling a PT device with MSI > enabled > > Hi, > > I am using the xen-3,4, and i am doing pass-through for a device with > MSI enabled (the device of-course supports MSI) to a domU with Windows > XP. When i disable and then re-enable the device using Windows device > manager, everything looks like it''s working fine, but the device seems > to malfunction. > After digging a little bit, i saw that the problem is, because the bit > of the interrupt-status (bit 3) in the status register (offset 0x6 in > the PCI config space) was turned on somehow - this proibably caused > the MSI to malfunction (because the IntX assertion was enabled - thus > disabling the MSI) - and this lead to the problem i experienced. > > Any ideas on how to solve this issue? > > Tom > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tom Rotenberg
2009-Nov-25 07:42 UTC
Re: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled
Hi, 1) It''s not display, i am using a wlan device (which Windows tends to disable and re-enable in certain cases) 2) I am using a 2.6.24 xen kernel I will do the check with the 2.6.18 xen kernel, and see if it''s responsible for this bug. On Wed, Nov 25, 2009 at 4:25 AM, Kamala Narasimhan <Kamala.Narasimhan@citrix.com> wrote:> > Couple of questions - > > 1) What kind of device are you referring to below? Display by any chance? > 2) Which tree is your dom0 kernel based on? > > There is a chance you are missing some msi patches in your dom0 kernel. You might want to do a quick search for msi patches in http://xenbits.xen.org/staging/linux-2.6.18-xen.hg?rev=msi and see if you are missing anything obvious. I recently encountered a similar issue in a slightly different situation, a subset of the problem was caused because of missing msi patches. > > Kamala > >> -----Original Message----- >> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel- >> bounces@lists.xensource.com] On Behalf Of Tom Rotenberg >> Sent: Tuesday, November 24, 2009 1:43 PM >> To: xen-devel@lists.xensource.com >> Subject: [Xen-devel] Bug when disabling/enabling a PT device with MSI >> enabled >> >> Hi, >> >> I am using the xen-3,4, and i am doing pass-through for a device with >> MSI enabled (the device of-course supports MSI) to a domU with Windows >> XP. When i disable and then re-enable the device using Windows device >> manager, everything looks like it''s working fine, but the device seems >> to malfunction. >> After digging a little bit, i saw that the problem is, because the bit >> of the interrupt-status (bit 3) in the status register (offset 0x6 in >> the PCI config space) was turned on somehow - this proibably caused >> the MSI to malfunction (because the IntX assertion was enabled - thus >> disabling the MSI) - and this lead to the problem i experienced. >> >> Any ideas on how to solve this issue? >> >> Tom >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tom Rotenberg
2009-Nov-25 08:27 UTC
Re: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled
Well, just tested it with the 2.6.18 latest kernel, and the bug still exists. Also, i think that iit''s more related to how qemu handles the "disable/enable" of domU for PT devices with MSI... but i don''t know where to start from? it looks like when re-enabling a PT device with MSI, we need qemu to re-initialize the device according to it''s original MSI state (and not to what it was resetted before). Does anyone has any clue about it? On Wed, Nov 25, 2009 at 4:25 AM, Kamala Narasimhan <Kamala.Narasimhan@citrix.com> wrote:> > Couple of questions - > > 1) What kind of device are you referring to below? Display by any chance? > 2) Which tree is your dom0 kernel based on? > > There is a chance you are missing some msi patches in your dom0 kernel. You might want to do a quick search for msi patches in http://xenbits.xen.org/staging/linux-2.6.18-xen.hg?rev=msi and see if you are missing anything obvious. I recently encountered a similar issue in a slightly different situation, a subset of the problem was caused because of missing msi patches. > > Kamala > >> -----Original Message----- >> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel- >> bounces@lists.xensource.com] On Behalf Of Tom Rotenberg >> Sent: Tuesday, November 24, 2009 1:43 PM >> To: xen-devel@lists.xensource.com >> Subject: [Xen-devel] Bug when disabling/enabling a PT device with MSI >> enabled >> >> Hi, >> >> I am using the xen-3,4, and i am doing pass-through for a device with >> MSI enabled (the device of-course supports MSI) to a domU with Windows >> XP. When i disable and then re-enable the device using Windows device >> manager, everything looks like it''s working fine, but the device seems >> to malfunction. >> After digging a little bit, i saw that the problem is, because the bit >> of the interrupt-status (bit 3) in the status register (offset 0x6 in >> the PCI config space) was turned on somehow - this proibably caused >> the MSI to malfunction (because the IntX assertion was enabled - thus >> disabling the MSI) - and this lead to the problem i experienced. >> >> Any ideas on how to solve this issue? >> >> Tom >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jiang, Yunhong
2009-Nov-25 09:10 UTC
RE: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled
This should be about qemu''s MSI support. The code is in qemu/hw/pt-msi.c for all MSI emulation. Windows XP does not support MSI if I remember correctly. SO this issue should be about MSI translation, i.e. guest is using IOAPIC while host is using MSI. Xen hypervisor will translate the MSI to guest''s IOAPIC. --jyh ________________________________________ From: xen-devel-bounces@lists.xensource.com [xen-devel-bounces@lists.xensource.com] On Behalf Of Tom Rotenberg [tom.rotenberg@gmail.com] Sent: Wednesday, November 25, 2009 1:27 AM To: Kamala Narasimhan Cc: xen-devel@lists.xensource.com Subject: Re: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled Well, just tested it with the 2.6.18 latest kernel, and the bug still exists. Also, i think that iit''s more related to how qemu handles the "disable/enable" of domU for PT devices with MSI... but i don''t know where to start from? it looks like when re-enabling a PT device with MSI, we need qemu to re-initialize the device according to it''s original MSI state (and not to what it was resetted before). Does anyone has any clue about it? On Wed, Nov 25, 2009 at 4:25 AM, Kamala Narasimhan <Kamala.Narasimhan@citrix.com> wrote:> > Couple of questions - > > 1) What kind of device are you referring to below? Display by any chance? > 2) Which tree is your dom0 kernel based on? > > There is a chance you are missing some msi patches in your dom0 kernel. You might want to do a quick search for msi patches in http://xenbits.xen.org/staging/linux-2.6.18-xen.hg?rev=msi and see if you are missing anything obvious. I recently encountered a similar issue in a slightly different situation, a subset of the problem was caused because of missing msi patches. > > Kamala > >> -----Original Message----- >> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel- >> bounces@lists.xensource.com] On Behalf Of Tom Rotenberg >> Sent: Tuesday, November 24, 2009 1:43 PM >> To: xen-devel@lists.xensource.com >> Subject: [Xen-devel] Bug when disabling/enabling a PT device with MSI >> enabled >> >> Hi, >> >> I am using the xen-3,4, and i am doing pass-through for a device with >> MSI enabled (the device of-course supports MSI) to a domU with Windows >> XP. When i disable and then re-enable the device using Windows device >> manager, everything looks like it''s working fine, but the device seems >> to malfunction. >> After digging a little bit, i saw that the problem is, because the bit >> of the interrupt-status (bit 3) in the status register (offset 0x6 in >> the PCI config space) was turned on somehow - this proibably caused >> the MSI to malfunction (because the IntX assertion was enabled - thus >> disabling the MSI) - and this lead to the problem i experienced. >> >> Any ideas on how to solve this issue? >> >> Tom >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jiang, Yunhong
2009-Nov-25 09:17 UTC
RE: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled
Sorry send out the mail too quickly. I realized it is MSI translation when I finished first section but forgot remove it, so please ignore first section. You only need care about MSI translation, the code is in hw/pass-through.c. You can check qemu''s changelog at a8d1bd66. --jyh ________________________________________ From: xen-devel-bounces@lists.xensource.com [xen-devel-bounces@lists.xensource.com] On Behalf Of Jiang, Yunhong [yunhong.jiang@intel.com] Sent: Wednesday, November 25, 2009 2:10 AM To: Tom Rotenberg; Kamala Narasimhan Cc: xen-devel@lists.xensource.com Subject: RE: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled This should be about qemu''s MSI support. The code is in qemu/hw/pt-msi.c for all MSI emulation. Windows XP does not support MSI if I remember correctly. SO this issue should be about MSI translation, i.e. guest is using IOAPIC while host is using MSI. Xen hypervisor will translate the MSI to guest''s IOAPIC. --jyh ________________________________________ From: xen-devel-bounces@lists.xensource.com [xen-devel-bounces@lists.xensource.com] On Behalf Of Tom Rotenberg [tom.rotenberg@gmail.com] Sent: Wednesday, November 25, 2009 1:27 AM To: Kamala Narasimhan Cc: xen-devel@lists.xensource.com Subject: Re: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled Well, just tested it with the 2.6.18 latest kernel, and the bug still exists. Also, i think that iit''s more related to how qemu handles the "disable/enable" of domU for PT devices with MSI... but i don''t know where to start from? it looks like when re-enabling a PT device with MSI, we need qemu to re-initialize the device according to it''s original MSI state (and not to what it was resetted before). Does anyone has any clue about it? On Wed, Nov 25, 2009 at 4:25 AM, Kamala Narasimhan <Kamala.Narasimhan@citrix.com> wrote:> > Couple of questions - > > 1) What kind of device are you referring to below? Display by any chance? > 2) Which tree is your dom0 kernel based on? > > There is a chance you are missing some msi patches in your dom0 kernel. You might want to do a quick search for msi patches in http://xenbits.xen.org/staging/linux-2.6.18-xen.hg?rev=msi and see if you are missing anything obvious. I recently encountered a similar issue in a slightly different situation, a subset of the problem was caused because of missing msi patches. > > Kamala > >> -----Original Message----- >> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel- >> bounces@lists.xensource.com] On Behalf Of Tom Rotenberg >> Sent: Tuesday, November 24, 2009 1:43 PM >> To: xen-devel@lists.xensource.com >> Subject: [Xen-devel] Bug when disabling/enabling a PT device with MSI >> enabled >> >> Hi, >> >> I am using the xen-3,4, and i am doing pass-through for a device with >> MSI enabled (the device of-course supports MSI) to a domU with Windows >> XP. When i disable and then re-enable the device using Windows device >> manager, everything looks like it''s working fine, but the device seems >> to malfunction. >> After digging a little bit, i saw that the problem is, because the bit >> of the interrupt-status (bit 3) in the status register (offset 0x6 in >> the PCI config space) was turned on somehow - this proibably caused >> the MSI to malfunction (because the IntX assertion was enabled - thus >> disabling the MSI) - and this lead to the problem i experienced. >> >> Any ideas on how to solve this issue? >> >> Tom >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tom Rotenberg
2009-Nov-25 09:29 UTC
Re: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled
Well, to try and see if the problem is msi related or not, i registered the device with the option of ''msi_translate=0'' to force the use of intx instead of MSI, and i still got the problem... so i guess it''s not related to the MSI. It looks like the IntX bit (bit 3 of status word) is runed on somehow outside qemu, and that''s the problem... could that be? On Wed, Nov 25, 2009 at 11:17 AM, Jiang, Yunhong <yunhong.jiang@intel.com> wrote:> Sorry send out the mail too quickly. > I realized it is MSI translation when I finished first section but forgot remove it, so please ignore first section. > You only need care about MSI translation, the code is in hw/pass-through.c. You can check qemu''s changelog at a8d1bd66. > > --jyh > ________________________________________ > From: xen-devel-bounces@lists.xensource.com [xen-devel-bounces@lists.xensource.com] On Behalf Of Jiang, Yunhong [yunhong.jiang@intel.com] > Sent: Wednesday, November 25, 2009 2:10 AM > To: Tom Rotenberg; Kamala Narasimhan > Cc: xen-devel@lists.xensource.com > Subject: RE: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled > > This should be about qemu''s MSI support. The code is in qemu/hw/pt-msi.c for all MSI emulation. > Windows XP does not support MSI if I remember correctly. SO this issue should be about MSI translation, i.e. guest is using IOAPIC while host is using MSI. Xen hypervisor will translate the MSI to guest''s IOAPIC. > > --jyh > ________________________________________ > From: xen-devel-bounces@lists.xensource.com [xen-devel-bounces@lists.xensource.com] On Behalf Of Tom Rotenberg [tom.rotenberg@gmail.com] > Sent: Wednesday, November 25, 2009 1:27 AM > To: Kamala Narasimhan > Cc: xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled > > Well, just tested it with the 2.6.18 latest kernel, and the bug still exists. > Also, i think that iit''s more related to how qemu handles the > "disable/enable" of domU for PT devices with MSI... but i don''t know > where to start from? it looks like when re-enabling a PT device with > MSI, we need qemu to re-initialize the device according to it''s > original MSI state (and not to what it was resetted before). > > Does anyone has any clue about it? > > On Wed, Nov 25, 2009 at 4:25 AM, Kamala Narasimhan > <Kamala.Narasimhan@citrix.com> wrote: >> >> Couple of questions - >> >> 1) What kind of device are you referring to below? Display by any chance? >> 2) Which tree is your dom0 kernel based on? >> >> There is a chance you are missing some msi patches in your dom0 kernel. You might want to do a quick search for msi patches in http://xenbits.xen.org/staging/linux-2.6.18-xen.hg?rev=msi and see if you are missing anything obvious. I recently encountered a similar issue in a slightly different situation, a subset of the problem was caused because of missing msi patches. >> >> Kamala >> >>> -----Original Message----- >>> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel- >>> bounces@lists.xensource.com] On Behalf Of Tom Rotenberg >>> Sent: Tuesday, November 24, 2009 1:43 PM >>> To: xen-devel@lists.xensource.com >>> Subject: [Xen-devel] Bug when disabling/enabling a PT device with MSI >>> enabled >>> >>> Hi, >>> >>> I am using the xen-3,4, and i am doing pass-through for a device with >>> MSI enabled (the device of-course supports MSI) to a domU with Windows >>> XP. When i disable and then re-enable the device using Windows device >>> manager, everything looks like it''s working fine, but the device seems >>> to malfunction. >>> After digging a little bit, i saw that the problem is, because the bit >>> of the interrupt-status (bit 3) in the status register (offset 0x6 in >>> the PCI config space) was turned on somehow - this proibably caused >>> the MSI to malfunction (because the IntX assertion was enabled - thus >>> disabling the MSI) - and this lead to the problem i experienced. >>> >>> Any ideas on how to solve this issue? >>> >>> Tom >>> >>> _______________________________________________ >>> Xen-devel mailing list >>> Xen-devel@lists.xensource.com >>> http://lists.xensource.com/xen-devel >> > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jiang, Yunhong
2009-Nov-25 09:38 UTC
RE: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled
If the msi_translate=0, why IntX bit is turned on is wrong? I assume guest should use intX in such situation. --jyh ________________________________________ From: Tom Rotenberg [tom.rotenberg@gmail.com] Sent: Wednesday, November 25, 2009 2:29 AM To: Jiang, Yunhong Cc: Kamala Narasimhan; xen-devel@lists.xensource.com Subject: Re: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled Well, to try and see if the problem is msi related or not, i registered the device with the option of ''msi_translate=0'' to force the use of intx instead of MSI, and i still got the problem... so i guess it''s not related to the MSI. It looks like the IntX bit (bit 3 of status word) is runed on somehow outside qemu, and that''s the problem... could that be? On Wed, Nov 25, 2009 at 11:17 AM, Jiang, Yunhong <yunhong.jiang@intel.com> wrote:> Sorry send out the mail too quickly. > I realized it is MSI translation when I finished first section but forgot remove it, so please ignore first section. > You only need care about MSI translation, the code is in hw/pass-through.c. You can check qemu''s changelog at a8d1bd66. > > --jyh > ________________________________________ > From: xen-devel-bounces@lists.xensource.com [xen-devel-bounces@lists.xensource.com] On Behalf Of Jiang, Yunhong [yunhong.jiang@intel.com] > Sent: Wednesday, November 25, 2009 2:10 AM > To: Tom Rotenberg; Kamala Narasimhan > Cc: xen-devel@lists.xensource.com > Subject: RE: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled > > This should be about qemu''s MSI support. The code is in qemu/hw/pt-msi.c for all MSI emulation. > Windows XP does not support MSI if I remember correctly. SO this issue should be about MSI translation, i.e. guest is using IOAPIC while host is using MSI. Xen hypervisor will translate the MSI to guest''s IOAPIC. > > --jyh > ________________________________________ > From: xen-devel-bounces@lists.xensource.com [xen-devel-bounces@lists.xensource.com] On Behalf Of Tom Rotenberg [tom.rotenberg@gmail.com] > Sent: Wednesday, November 25, 2009 1:27 AM > To: Kamala Narasimhan > Cc: xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled > > Well, just tested it with the 2.6.18 latest kernel, and the bug still exists. > Also, i think that iit''s more related to how qemu handles the > "disable/enable" of domU for PT devices with MSI... but i don''t know > where to start from? it looks like when re-enabling a PT device with > MSI, we need qemu to re-initialize the device according to it''s > original MSI state (and not to what it was resetted before). > > Does anyone has any clue about it? > > On Wed, Nov 25, 2009 at 4:25 AM, Kamala Narasimhan > <Kamala.Narasimhan@citrix.com> wrote: >> >> Couple of questions - >> >> 1) What kind of device are you referring to below? Display by any chance? >> 2) Which tree is your dom0 kernel based on? >> >> There is a chance you are missing some msi patches in your dom0 kernel. You might want to do a quick search for msi patches in http://xenbits.xen.org/staging/linux-2.6.18-xen.hg?rev=msi and see if you are missing anything obvious. I recently encountered a similar issue in a slightly different situation, a subset of the problem was caused because of missing msi patches. >> >> Kamala >> >>> -----Original Message----- >>> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel- >>> bounces@lists.xensource.com] On Behalf Of Tom Rotenberg >>> Sent: Tuesday, November 24, 2009 1:43 PM >>> To: xen-devel@lists.xensource.com >>> Subject: [Xen-devel] Bug when disabling/enabling a PT device with MSI >>> enabled >>> >>> Hi, >>> >>> I am using the xen-3,4, and i am doing pass-through for a device with >>> MSI enabled (the device of-course supports MSI) to a domU with Windows >>> XP. When i disable and then re-enable the device using Windows device >>> manager, everything looks like it''s working fine, but the device seems >>> to malfunction. >>> After digging a little bit, i saw that the problem is, because the bit >>> of the interrupt-status (bit 3) in the status register (offset 0x6 in >>> the PCI config space) was turned on somehow - this proibably caused >>> the MSI to malfunction (because the IntX assertion was enabled - thus >>> disabling the MSI) - and this lead to the problem i experienced. >>> >>> Any ideas on how to solve this issue? >>> >>> Tom >>> >>> _______________________________________________ >>> Xen-devel mailing list >>> Xen-devel@lists.xensource.com >>> http://lists.xensource.com/xen-devel >> > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tom Rotenberg
2009-Nov-25 09:54 UTC
Re: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled
Well, you are correct... however, the situation is that the bgefore the disable the intx bit was turned off, and only after disable and then re-enable it was turned on, and that''s the strange part of it... On Wed, Nov 25, 2009 at 11:38 AM, Jiang, Yunhong <yunhong.jiang@intel.com> wrote:> If the msi_translate=0, why IntX bit is turned on is wrong? I assume guest should use intX in such situation. > > --jyh > ________________________________________ > From: Tom Rotenberg [tom.rotenberg@gmail.com] > Sent: Wednesday, November 25, 2009 2:29 AM > To: Jiang, Yunhong > Cc: Kamala Narasimhan; xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled > > Well, to try and see if the problem is msi related or not, i > registered the device with the option of ''msi_translate=0'' to force > the use of intx instead of MSI, and i still got the problem... so i > guess it''s not related to the MSI. > > It looks like the IntX bit (bit 3 of status word) is runed on somehow > outside qemu, and that''s the problem... could that be? > > On Wed, Nov 25, 2009 at 11:17 AM, Jiang, Yunhong > <yunhong.jiang@intel.com> wrote: >> Sorry send out the mail too quickly. >> I realized it is MSI translation when I finished first section but forgot remove it, so please ignore first section. >> You only need care about MSI translation, the code is in hw/pass-through.c. You can check qemu''s changelog at a8d1bd66. >> >> --jyh >> ________________________________________ >> From: xen-devel-bounces@lists.xensource.com [xen-devel-bounces@lists.xensource.com] On Behalf Of Jiang, Yunhong [yunhong.jiang@intel.com] >> Sent: Wednesday, November 25, 2009 2:10 AM >> To: Tom Rotenberg; Kamala Narasimhan >> Cc: xen-devel@lists.xensource.com >> Subject: RE: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled >> >> This should be about qemu''s MSI support. The code is in qemu/hw/pt-msi.c for all MSI emulation. >> Windows XP does not support MSI if I remember correctly. SO this issue should be about MSI translation, i.e. guest is using IOAPIC while host is using MSI. Xen hypervisor will translate the MSI to guest''s IOAPIC. >> >> --jyh >> ________________________________________ >> From: xen-devel-bounces@lists.xensource.com [xen-devel-bounces@lists.xensource.com] On Behalf Of Tom Rotenberg [tom.rotenberg@gmail.com] >> Sent: Wednesday, November 25, 2009 1:27 AM >> To: Kamala Narasimhan >> Cc: xen-devel@lists.xensource.com >> Subject: Re: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled >> >> Well, just tested it with the 2.6.18 latest kernel, and the bug still exists. >> Also, i think that iit''s more related to how qemu handles the >> "disable/enable" of domU for PT devices with MSI... but i don''t know >> where to start from? it looks like when re-enabling a PT device with >> MSI, we need qemu to re-initialize the device according to it''s >> original MSI state (and not to what it was resetted before). >> >> Does anyone has any clue about it? >> >> On Wed, Nov 25, 2009 at 4:25 AM, Kamala Narasimhan >> <Kamala.Narasimhan@citrix.com> wrote: >>> >>> Couple of questions - >>> >>> 1) What kind of device are you referring to below? Display by any chance? >>> 2) Which tree is your dom0 kernel based on? >>> >>> There is a chance you are missing some msi patches in your dom0 kernel. You might want to do a quick search for msi patches in http://xenbits.xen.org/staging/linux-2.6.18-xen.hg?rev=msi and see if you are missing anything obvious. I recently encountered a similar issue in a slightly different situation, a subset of the problem was caused because of missing msi patches. >>> >>> Kamala >>> >>>> -----Original Message----- >>>> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel- >>>> bounces@lists.xensource.com] On Behalf Of Tom Rotenberg >>>> Sent: Tuesday, November 24, 2009 1:43 PM >>>> To: xen-devel@lists.xensource.com >>>> Subject: [Xen-devel] Bug when disabling/enabling a PT device with MSI >>>> enabled >>>> >>>> Hi, >>>> >>>> I am using the xen-3,4, and i am doing pass-through for a device with >>>> MSI enabled (the device of-course supports MSI) to a domU with Windows >>>> XP. When i disable and then re-enable the device using Windows device >>>> manager, everything looks like it''s working fine, but the device seems >>>> to malfunction. >>>> After digging a little bit, i saw that the problem is, because the bit >>>> of the interrupt-status (bit 3) in the status register (offset 0x6 in >>>> the PCI config space) was turned on somehow - this proibably caused >>>> the MSI to malfunction (because the IntX assertion was enabled - thus >>>> disabling the MSI) - and this lead to the problem i experienced. >>>> >>>> Any ideas on how to solve this issue? >>>> >>>> Tom >>>> >>>> _______________________________________________ >>>> Xen-devel mailing list >>>> Xen-devel@lists.xensource.com >>>> http://lists.xensource.com/xen-devel >>> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jiang, Yunhong
2009-Nov-25 14:47 UTC
RE: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled
Tom Rotenberg wrote:> Well, you are correct... however, the situation is that the bgefore > the disable the intx bit was turned off, and only after disable and > then re-enable it was turned on, and that''s the strange part of it...You mean even if the msi_translate == 0? That''s strange. If this happens only under msi_translate, it is reasonable and maybe something wrong. So can you give me a confirmation? --jyh> > On Wed, Nov 25, 2009 at 11:38 AM, Jiang, Yunhong > <yunhong.jiang@intel.com> wrote: >> If the msi_translate=0, why IntX bit is turned on is wrong? > I assume guest should use intX in such situation. >> >> --jyh >> ________________________________________ >> From: Tom Rotenberg [tom.rotenberg@gmail.com] >> Sent: Wednesday, November 25, 2009 2:29 AM >> To: Jiang, Yunhong >> Cc: Kamala Narasimhan; xen-devel@lists.xensource.com >> Subject: Re: [Xen-devel] Bug when disabling/enabling a PT device >> with MSI enabled >> >> Well, to try and see if the problem is msi related or not, i >> registered the device with the option of ''msi_translate=0'' to force >> the use of intx instead of MSI, and i still got the problem... so i >> guess it''s not related to the MSI. >> >> It looks like the IntX bit (bit 3 of status word) is runed on somehow >> outside qemu, and that''s the problem... could that be? >> >> On Wed, Nov 25, 2009 at 11:17 AM, Jiang, Yunhong >> <yunhong.jiang@intel.com> wrote: >>> Sorry send out the mail too quickly. >>> I realized it is MSI translation when I finished first > section but forgot remove it, so please ignore first section. >>> You only need care about MSI translation, the code is in > hw/pass-through.c. You can check qemu''s changelog at a8d1bd66. >>> >>> --jyh >>> ________________________________________ >>> From: xen-devel-bounces@lists.xensource.com > [xen-devel-bounces@lists.xensource.com] On Behalf Of Jiang, > Yunhong [yunhong.jiang@intel.com] >>> Sent: Wednesday, November 25, 2009 2:10 AM >>> To: Tom Rotenberg; Kamala Narasimhan >>> Cc: xen-devel@lists.xensource.com >>> Subject: RE: [Xen-devel] Bug when disabling/enabling a PT device >>> with MSI enabled >>> >>> This should be about qemu''s MSI support. The code is in > qemu/hw/pt-msi.c for all MSI emulation. >>> Windows XP does not support MSI if I remember correctly. SO > this issue should be about MSI translation, i.e. guest is > using IOAPIC while host is using MSI. Xen hypervisor will > translate the MSI to guest''s IOAPIC. >>> >>> --jyh >>> ________________________________________ >>> From: xen-devel-bounces@lists.xensource.com > [xen-devel-bounces@lists.xensource.com] On Behalf Of Tom > Rotenberg [tom.rotenberg@gmail.com] >>> Sent: Wednesday, November 25, 2009 1:27 AM >>> To: Kamala Narasimhan >>> Cc: xen-devel@lists.xensource.com >>> Subject: Re: [Xen-devel] Bug when disabling/enabling a PT device >>> with MSI enabled >>> >>> Well, just tested it with the 2.6.18 latest kernel, and the bug >>> still exists. Also, i think that iit''s more related to how qemu >>> handles the "disable/enable" of domU for PT devices with MSI... but >>> i don''t know where to start from? it looks like when re-enabling a >>> PT device with MSI, we need qemu to re-initialize the device >>> according to it''s original MSI state (and not to what it was >>> resetted before). >>> >>> Does anyone has any clue about it? >>> >>> On Wed, Nov 25, 2009 at 4:25 AM, Kamala Narasimhan >>> <Kamala.Narasimhan@citrix.com> wrote: >>>> >>>> Couple of questions - >>>> >>>> 1) What kind of device are you referring to below? Display by any >>>> chance? 2) Which tree is your dom0 kernel based on? >>>> >>>> There is a chance you are missing some msi patches in your > dom0 kernel. You might want to do a quick search for msi > patches in > http://xenbits.xen.org/staging/linux-2.6.18-xen.hg?rev=msi and > see if you are missing anything obvious. I recently > encountered a similar issue in a slightly different situation, > a subset of the problem was caused because of missing msi patches. >>>> >>>> Kamala >>>> >>>>> -----Original Message----- >>>>> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel- >>>>> bounces@lists.xensource.com] On Behalf Of Tom Rotenberg >>>>> Sent: Tuesday, November 24, 2009 1:43 PM >>>>> To: xen-devel@lists.xensource.com >>>>> Subject: [Xen-devel] Bug when disabling/enabling a PT device with >>>>> MSI enabled >>>>> >>>>> Hi, >>>>> >>>>> I am using the xen-3,4, and i am doing pass-through for a device >>>>> with MSI enabled (the device of-course supports MSI) to a domU >>>>> with Windows XP. When i disable and then re-enable the device >>>>> using Windows device manager, everything looks like it''s working >>>>> fine, but the device seems to malfunction. After digging a little >>>>> bit, i saw that the problem is, because the bit of the >>>>> interrupt-status (bit 3) in the status register (offset 0x6 in >>>>> the PCI config space) was turned on somehow - this proibably >>>>> caused the MSI to malfunction (because the IntX assertion was >>>>> enabled - thus disabling the MSI) - and this lead to the problem >>>>> i experienced. >>>>> >>>>> Any ideas on how to solve this issue? >>>>> >>>>> Tom >>>>> >>>>> _______________________________________________ >>>>> Xen-devel mailing list >>>>> Xen-devel@lists.xensource.com >>>>> http://lists.xensource.com/xen-devel >>>> >>> >>> _______________________________________________ >>> Xen-devel mailing list >>> Xen-devel@lists.xensource.com >>> http://lists.xensource.com/xen-devel >>> >>> _______________________________________________ >>> Xen-devel mailing list >>> Xen-devel@lists.xensource.com >>> http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tom Rotenberg
2009-Nov-25 14:50 UTC
Re: [Xen-devel] Bug when disabling/enabling a PT device with MSI enabled
It happens also when msi_translte = 0 , and i think that it''s not related to the MSI at all... i also checked other devices which use MSI (like the audio device), and they worked fine. You can take a look at the new mail i sent, which incorporates more details about this problem. On Wed, Nov 25, 2009 at 4:47 PM, Jiang, Yunhong <yunhong.jiang@intel.com> wrote:> Tom Rotenberg wrote: >> Well, you are correct... however, the situation is that the bgefore >> the disable the intx bit was turned off, and only after disable and >> then re-enable it was turned on, and that''s the strange part of it... > > You mean even if the msi_translate == 0? That''s strange. If this happens only under msi_translate, it is reasonable and maybe something wrong. So can you give me a confirmation? > > --jyh > >> >> On Wed, Nov 25, 2009 at 11:38 AM, Jiang, Yunhong >> <yunhong.jiang@intel.com> wrote: >>> If the msi_translate=0, why IntX bit is turned on is wrong? >> I assume guest should use intX in such situation. >>> >>> --jyh >>> ________________________________________ >>> From: Tom Rotenberg [tom.rotenberg@gmail.com] >>> Sent: Wednesday, November 25, 2009 2:29 AM >>> To: Jiang, Yunhong >>> Cc: Kamala Narasimhan; xen-devel@lists.xensource.com >>> Subject: Re: [Xen-devel] Bug when disabling/enabling a PT device >>> with MSI enabled >>> >>> Well, to try and see if the problem is msi related or not, i >>> registered the device with the option of ''msi_translate=0'' to force >>> the use of intx instead of MSI, and i still got the problem... so i >>> guess it''s not related to the MSI. >>> >>> It looks like the IntX bit (bit 3 of status word) is runed on somehow >>> outside qemu, and that''s the problem... could that be? >>> >>> On Wed, Nov 25, 2009 at 11:17 AM, Jiang, Yunhong >>> <yunhong.jiang@intel.com> wrote: >>>> Sorry send out the mail too quickly. >>>> I realized it is MSI translation when I finished first >> section but forgot remove it, so please ignore first section. >>>> You only need care about MSI translation, the code is in >> hw/pass-through.c. You can check qemu''s changelog at a8d1bd66. >>>> >>>> --jyh >>>> ________________________________________ >>>> From: xen-devel-bounces@lists.xensource.com >> [xen-devel-bounces@lists.xensource.com] On Behalf Of Jiang, >> Yunhong [yunhong.jiang@intel.com] >>>> Sent: Wednesday, November 25, 2009 2:10 AM >>>> To: Tom Rotenberg; Kamala Narasimhan >>>> Cc: xen-devel@lists.xensource.com >>>> Subject: RE: [Xen-devel] Bug when disabling/enabling a PT device >>>> with MSI enabled >>>> >>>> This should be about qemu''s MSI support. The code is in >> qemu/hw/pt-msi.c for all MSI emulation. >>>> Windows XP does not support MSI if I remember correctly. SO >> this issue should be about MSI translation, i.e. guest is >> using IOAPIC while host is using MSI. Xen hypervisor will >> translate the MSI to guest''s IOAPIC. >>>> >>>> --jyh >>>> ________________________________________ >>>> From: xen-devel-bounces@lists.xensource.com >> [xen-devel-bounces@lists.xensource.com] On Behalf Of Tom >> Rotenberg [tom.rotenberg@gmail.com] >>>> Sent: Wednesday, November 25, 2009 1:27 AM >>>> To: Kamala Narasimhan >>>> Cc: xen-devel@lists.xensource.com >>>> Subject: Re: [Xen-devel] Bug when disabling/enabling a PT device >>>> with MSI enabled >>>> >>>> Well, just tested it with the 2.6.18 latest kernel, and the bug >>>> still exists. Also, i think that iit''s more related to how qemu >>>> handles the "disable/enable" of domU for PT devices with MSI... but >>>> i don''t know where to start from? it looks like when re-enabling a >>>> PT device with MSI, we need qemu to re-initialize the device >>>> according to it''s original MSI state (and not to what it was >>>> resetted before). >>>> >>>> Does anyone has any clue about it? >>>> >>>> On Wed, Nov 25, 2009 at 4:25 AM, Kamala Narasimhan >>>> <Kamala.Narasimhan@citrix.com> wrote: >>>>> >>>>> Couple of questions - >>>>> >>>>> 1) What kind of device are you referring to below? Display by any >>>>> chance? 2) Which tree is your dom0 kernel based on? >>>>> >>>>> There is a chance you are missing some msi patches in your >> dom0 kernel. You might want to do a quick search for msi >> patches in >> http://xenbits.xen.org/staging/linux-2.6.18-xen.hg?rev=msi and >> see if you are missing anything obvious. I recently >> encountered a similar issue in a slightly different situation, >> a subset of the problem was caused because of missing msi patches. >>>>> >>>>> Kamala >>>>> >>>>>> -----Original Message----- >>>>>> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel- >>>>>> bounces@lists.xensource.com] On Behalf Of Tom Rotenberg >>>>>> Sent: Tuesday, November 24, 2009 1:43 PM >>>>>> To: xen-devel@lists.xensource.com >>>>>> Subject: [Xen-devel] Bug when disabling/enabling a PT device with >>>>>> MSI enabled >>>>>> >>>>>> Hi, >>>>>> >>>>>> I am using the xen-3,4, and i am doing pass-through for a device >>>>>> with MSI enabled (the device of-course supports MSI) to a domU >>>>>> with Windows XP. When i disable and then re-enable the device >>>>>> using Windows device manager, everything looks like it''s working >>>>>> fine, but the device seems to malfunction. After digging a little >>>>>> bit, i saw that the problem is, because the bit of the >>>>>> interrupt-status (bit 3) in the status register (offset 0x6 in >>>>>> the PCI config space) was turned on somehow - this proibably >>>>>> caused the MSI to malfunction (because the IntX assertion was >>>>>> enabled - thus disabling the MSI) - and this lead to the problem >>>>>> i experienced. >>>>>> >>>>>> Any ideas on how to solve this issue? >>>>>> >>>>>> Tom >>>>>> >>>>>> _______________________________________________ >>>>>> Xen-devel mailing list >>>>>> Xen-devel@lists.xensource.com >>>>>> http://lists.xensource.com/xen-devel >>>>> >>>> >>>> _______________________________________________ >>>> Xen-devel mailing list >>>> Xen-devel@lists.xensource.com >>>> http://lists.xensource.com/xen-devel >>>> >>>> _______________________________________________ >>>> Xen-devel mailing list >>>> Xen-devel@lists.xensource.com >>>> http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel