lspci shows MSI enabled for PCI device. PCI passthrough works fine.
However, as soon as the MSI driver for card is insmodded, kernel panics.
This is on xen-unstable. Tried the same with xen-3.3.0 which is
supposed to have MSI passthrough, but the same guest shows MSI as disabled.
Any else seen this bug, or know of a workaround ?
Trace is as follows :
------------[ cut here ]------------
kernel BUG at
/usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:809!
invalid opcode: 0000 [#1]
SMP
Modules linked in: nfemsg nfdvnet ipv6 binfmt_misc dm_mod nfe usbcore
ext3 jbd processor fuse
CPU: 0
EIP: 0061:[<c02487f5>] Tainted: GF VLI
EFLAGS: 00210097 (2.6.18.8-xen #2)
EIP is at evtchn_get_xen_pirq+0x35/0x40
eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000
esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac
ds: 007b es: 007b ss: 0069
Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 task.ti=ed384000)
Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 00000000
00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000
Call Trace:
[<c0248aef>] startup_pirq+0x3f/0x250
[<c0150b50>] setup_irq+0x160/0x1b0
[<ee0dd270>] nfe_interrupt_handler+0x0/0x30 [nfemsg]
[<c0150c43>] request_irq+0xa3/0xc0
[<ee02c8ad>] nfemsg_module_init+0x8ad/0x133e [nfemsg]
[<c030531b>] cond_resched+0x2b/0x40
[<c0305369>] wait_for_completion+0x19/0xf0
[<c0142818>] sys_init_module+0x148/0x1b50
[<c010595f>] syscall_call+0x7/0xb
Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 44 c0 89
d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 <0f> 0b
29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1
EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP 0069:ed385dac
### card [0] start: host progs ###
-bash-3.2#
Message from syslogd@drake at Sep 19 15:36:44 ...
kernel: ------------[ cut here ]------------
Message from syslogd@drake at Sep 19 15:36:44 ...
kernel: kernel BUG at
/usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:809!
Message from syslogd@drake at Sep 19 15:36:44 ...
kernel: invalid opcode: 0000 [#1]
Message from syslogd@drake at Sep 19 15:36:44 ...
kernel: SMP
Message from syslogd@drake at Sep 19 15:36:44 ...
kernel: CPU: 0
Message from syslogd@drake at Sep 19 15:36:44 ...
kernel: EIP is at evtchn_get_xen_pirq+0x35/0x40
Message from syslogd@drake at Sep 19 15:36:44 ...
kernel: eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000
Message from syslogd@drake at Sep 19 15:36:44 ...
kernel: esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac
Message from syslogd@drake at Sep 19 15:36:44 ...
kernel: ds: 007b es: 007b ss: 0069
Message from syslogd@drake at Sep 19 15:36:44 ...
kernel: Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100
task.ti=ed384000)
Message from syslogd@drake at Sep 19 15:36:44 ...
kernel: Stack: c0248aef 00000000 00000000 00000000 00000000 00000000
00000000 00000000
Message from syslogd@drake at Sep 19 15:36:44 ...
kernel: 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000
Message from syslogd@drake at Sep 19 15:36:44 ...
kernel: 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000
Message from syslogd@drake at Sep 19 15:36:44 ...
kernel: Call Trace:
Message from syslogd@drake at Sep 19 15:36:44 ...
kernel: Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0
44 c0 89 d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3
<0f> 0b 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1
Message from syslogd@drake at Sep 19 15:36:44 ...
kernel: EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP
0069:ed385dac
--
As long as the music''s loud enough, we won''t hear the world
falling apart.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
On 3.3.0 you need to specify msi=1 on Xen''s command line to enable MSI. In xen-unstable, MSI support is always enabled. I changed a check in evtchn_get_xen_pirq() into a BUG_ON() as it looks to me like it should never trigger. Jan Beulich was the original authour of that function -- cc''ed in case he can indicate whether I''ve actually broken the function. :-) -- Keir On 19/9/08 20:53, "Anish Bhatt" <anish@cc.gatech.edu> wrote:> lspci shows MSI enabled for PCI device. PCI passthrough works fine. > However, as soon as the MSI driver for card is insmodded, kernel panics. > This is on xen-unstable. Tried the same with xen-3.3.0 which is > supposed to have MSI passthrough, but the same guest shows MSI as disabled. > Any else seen this bug, or know of a workaround ? > > Trace is as follows : > > ------------[ cut here ]------------ > kernel BUG at >/usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:809> !> invalid opcode: 0000 [#1] > SMP > Modules linked in: nfemsg nfdvnet ipv6 binfmt_misc dm_mod nfe usbcore > ext3 jbd processor fuse > CPU: 0 > EIP: 0061:[<c02487f5>] Tainted: GF VLI > EFLAGS: 00210097 (2.6.18.8-xen #2) > EIP is at evtchn_get_xen_pirq+0x35/0x40 > eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 > esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac > ds: 007b es: 007b ss: 0069 > Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 task.ti=ed384000) > Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 00000000 > 00000000 > 00000000 00000000 00000000 00000000 00000000 00000000 00000000 > 00000000 > 00000000 00000000 00000000 00000000 00000000 00000000 00000000 > 00000000 > Call Trace: > [<c0248aef>] startup_pirq+0x3f/0x250 > [<c0150b50>] setup_irq+0x160/0x1b0 > [<ee0dd270>] nfe_interrupt_handler+0x0/0x30 [nfemsg] > [<c0150c43>] request_irq+0xa3/0xc0 > [<ee02c8ad>] nfemsg_module_init+0x8ad/0x133e [nfemsg] > [<c030531b>] cond_resched+0x2b/0x40 > [<c0305369>] wait_for_completion+0x19/0xf0 > [<c0142818>] sys_init_module+0x148/0x1b50 > [<c010595f>] syscall_call+0x7/0xb > Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 44 c0 89 > d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 <0f> 0b > 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 > EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP 0069:ed385dac > ### card [0] start: host progs ### > -bash-3.2# > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: ------------[ cut here ]------------ > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: kernel BUG at >/usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:809> !> > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: invalid opcode: 0000 [#1] > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: SMP > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: CPU: 0 > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: EIP is at evtchn_get_xen_pirq+0x35/0x40 > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: ds: 007b es: 007b ss: 0069 > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 > task.ti=ed384000) > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 > 00000000 00000000 > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: 00000000 00000000 00000000 00000000 00000000 00000000 > 00000000 00000000 > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: 00000000 00000000 00000000 00000000 00000000 00000000 > 00000000 00000000 > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: Call Trace: > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 > 44 c0 89 d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 > <0f> 0b 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP > 0069:ed385dac_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Are you using MSI or MSI-X? I remember there are some bugfixes related to MSI-X recently. What are the changsets you are using for xen and PV kernel? Shan Haitao -----Original Message----- From: Anish Bhatt [mailto:anish@gatech.edu] Sent: 2008年9月21日 1:16 To: Keir Fraser Cc: xen-devel@lists.xensource.com; Shan, Haitao; Jan Beulich Subject: Re: [Xen-devel] MSI causing softpanics in guest I have put msi=1 in grub in 3.3 & unstable, don''t know why its still not working though.. -anish Keir Fraser wrote:> On 3.3.0 you need to specify msi=1 on Xen''s command line to enable MSI. In > xen-unstable, MSI support is always enabled. > > I changed a check in evtchn_get_xen_pirq() into a BUG_ON() as it looks to me > like it should never trigger. Jan Beulich was the original authour of that > function -- cc''ed in case he can indicate whether I''ve actually broken the > function. :-) > > -- Keir > > On 19/9/08 20:53, "Anish Bhatt" <anish@cc.gatech.edu> wrote: > > >> lspci shows MSI enabled for PCI device. PCI passthrough works fine. >> However, as soon as the MSI driver for card is insmodded, kernel panics. >> This is on xen-unstable. Tried the same with xen-3.3.0 which is >> supposed to have MSI passthrough, but the same guest shows MSI as disabled. >> Any else seen this bug, or know of a workaround ? >> >> Trace is as follows : >> >> ------------[ cut here ]------------ >> kernel BUG at >> >> > /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:809> > ! > >> invalid opcode: 0000 [#1] >> SMP >> Modules linked in: nfemsg nfdvnet ipv6 binfmt_misc dm_mod nfe usbcore >> ext3 jbd processor fuse >> CPU: 0 >> EIP: 0061:[<c02487f5>] Tainted: GF VLI >> EFLAGS: 00210097 (2.6.18.8-xen #2) >> EIP is at evtchn_get_xen_pirq+0x35/0x40 >> eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >> esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >> ds: 007b es: 007b ss: 0069 >> Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 task.ti=ed384000) >> Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 00000000 >> 00000000 >> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >> 00000000 >> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >> 00000000 >> Call Trace: >> [<c0248aef>] startup_pirq+0x3f/0x250 >> [<c0150b50>] setup_irq+0x160/0x1b0 >> [<ee0dd270>] nfe_interrupt_handler+0x0/0x30 [nfemsg] >> [<c0150c43>] request_irq+0xa3/0xc0 >> [<ee02c8ad>] nfemsg_module_init+0x8ad/0x133e [nfemsg] >> [<c030531b>] cond_resched+0x2b/0x40 >> [<c0305369>] wait_for_completion+0x19/0xf0 >> [<c0142818>] sys_init_module+0x148/0x1b50 >> [<c010595f>] syscall_call+0x7/0xb >> Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 44 c0 89 >> d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 <0f> 0b >> 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >> EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP 0069:ed385dac >> ### card [0] start: host progs ### >> -bash-3.2# >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: ------------[ cut here ]------------ >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: kernel BUG at >> >> > /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:809> > ! > >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: invalid opcode: 0000 [#1] >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: SMP >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: CPU: 0 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: EIP is at evtchn_get_xen_pirq+0x35/0x40 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: ds: 007b es: 007b ss: 0069 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 >> task.ti=ed384000) >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 >> 00000000 00000000 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >> 00000000 00000000 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >> 00000000 00000000 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: Call Trace: >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 >> 44 c0 89 d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 >> <0f> 0b 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP >> 0069:ed385dac >> > > >-- As long as the music''s loud enough, we won''t hear the world falling apart. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Seems it is not possible to trigger this bug_on, although I do not quite understand this part of code. I think you have invoked pci_enable_msi before request_irq, right? Does pci_enable_msi return correctly? Shan Haitao -----Original Message----- From: Anish Bhatt [mailto:anish@gatech.edu] Sent: 2008年9月22日 2:47 To: Shan, Haitao Cc: Keir Fraser; xen-devel@lists.xensource.com; Jan Beulich Subject: Re: [Xen-devel] MSI causing softpanics in guest I''m using MSI. I''m using the same 2.6.18 kernel for xen & PV, changeset is as follows : (XEN) Latest ChangeSet: Wed Sep 17 14:16:02 2008 +0100 18510:694b7daa353c -Anish Shan, Haitao wrote:> Are you using MSI or MSI-X? > I remember there are some bugfixes related to MSI-X recently. What are the changsets you are using for xen and PV kernel? > > Shan Haitao > > -----Original Message----- > From: Anish Bhatt [mailto:anish@gatech.edu] > Sent: 2008年9月21日 1:16 > To: Keir Fraser > Cc: xen-devel@lists.xensource.com; Shan, Haitao; Jan Beulich > Subject: Re: [Xen-devel] MSI causing softpanics in guest > > I have put msi=1 in grub in 3.3 & unstable, don''t know why its still not > working though.. > -anish > > Keir Fraser wrote: >> On 3.3.0 you need to specify msi=1 on Xen''s command line to enable MSI. In >> xen-unstable, MSI support is always enabled. >> >> I changed a check in evtchn_get_xen_pirq() into a BUG_ON() as it looks to me >> like it should never trigger. Jan Beulich was the original authour of that >> function -- cc''ed in case he can indicate whether I''ve actually broken the >> function. :-) >> >> -- Keir >> >> On 19/9/08 20:53, "Anish Bhatt" <anish@cc.gatech.edu> wrote: >> >> >>> lspci shows MSI enabled for PCI device. PCI passthrough works fine. >>> However, as soon as the MSI driver for card is insmodded, kernel panics. >>> This is on xen-unstable. Tried the same with xen-3.3.0 which is >>> supposed to have MSI passthrough, but the same guest shows MSI as disabled. >>> Any else seen this bug, or know of a workaround ? >>> >>> Trace is as follows : >>> >>> ------------[ cut here ]------------ >>> kernel BUG at >>> >>> >> /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:809> >> ! >> >>> invalid opcode: 0000 [#1] >>> SMP >>> Modules linked in: nfemsg nfdvnet ipv6 binfmt_misc dm_mod nfe usbcore >>> ext3 jbd processor fuse >>> CPU: 0 >>> EIP: 0061:[<c02487f5>] Tainted: GF VLI >>> EFLAGS: 00210097 (2.6.18.8-xen #2) >>> EIP is at evtchn_get_xen_pirq+0x35/0x40 >>> eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >>> esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >>> ds: 007b es: 007b ss: 0069 >>> Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 task.ti=ed384000) >>> Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 00000000 >>> 00000000 >>> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >>> 00000000 >>> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >>> 00000000 >>> Call Trace: >>> [<c0248aef>] startup_pirq+0x3f/0x250 >>> [<c0150b50>] setup_irq+0x160/0x1b0 >>> [<ee0dd270>] nfe_interrupt_handler+0x0/0x30 [nfemsg] >>> [<c0150c43>] request_irq+0xa3/0xc0 >>> [<ee02c8ad>] nfemsg_module_init+0x8ad/0x133e [nfemsg] >>> [<c030531b>] cond_resched+0x2b/0x40 >>> [<c0305369>] wait_for_completion+0x19/0xf0 >>> [<c0142818>] sys_init_module+0x148/0x1b50 >>> [<c010595f>] syscall_call+0x7/0xb >>> Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 44 c0 89 >>> d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 <0f> 0b >>> 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >>> EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP 0069:ed385dac >>> ### card [0] start: host progs ### >>> -bash-3.2# >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: ------------[ cut here ]------------ >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: kernel BUG at >>> >>> >> /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:809> >> ! >> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: invalid opcode: 0000 [#1] >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: SMP >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: CPU: 0 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: EIP is at evtchn_get_xen_pirq+0x35/0x40 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: ds: 007b es: 007b ss: 0069 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 >>> task.ti=ed384000) >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 >>> 00000000 00000000 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >>> 00000000 00000000 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >>> 00000000 00000000 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: Call Trace: >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 >>> 44 c0 89 d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 >>> <0f> 0b 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP >>> 0069:ed385dac >>> >> >> > > > -- > As long as the music''s loud enough, we won''t hear the world falling apart. >-- As long as the music''s loud enough, we won''t hear the world falling apart. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>>> Keir Fraser <keir.fraser@eu.citrix.com> 20.09.08 10:15 >>> >On 3.3.0 you need to specify msi=1 on Xen''s command line to enable MSI. In >xen-unstable, MSI support is always enabled. > >I changed a check in evtchn_get_xen_pirq() into a BUG_ON() as it looks to me >like it should never trigger. Jan Beulich was the original authour of that >function -- cc''ed in case he can indicate whether I''ve actually broken the >function. :-)No, I think that BUG_ON() you added is valid. It definitely is in the context of startup_pirq(). I''d be curious to know what the type of that irq really is, but that cannot be determined from the backtrace alone. Anish, could you perhaps add a simple printk() before the BUG_ON()? Unfortunately the driver in question does not appear to be part of the tree, so we can''t even check what it does prior to calling request_irq()... -- Keir On 19/9/08 20:53, "Anish Bhatt" <anish@cc.gatech.edu> wrote:> lspci shows MSI enabled for PCI device. PCI passthrough works fine. > However, as soon as the MSI driver for card is insmodded, kernel panics. > This is on xen-unstable. Tried the same with xen-3.3.0 which is > supposed to have MSI passthrough, but the same guest shows MSI as disabled. > Any else seen this bug, or know of a workaround ? > > Trace is as follows : > > ------------[ cut here ]------------ > kernel BUG at >/usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:809> !> invalid opcode: 0000 [#1] > SMP > Modules linked in: nfemsg nfdvnet ipv6 binfmt_misc dm_mod nfe usbcore > ext3 jbd processor fuse > CPU: 0 > EIP: 0061:[<c02487f5>] Tainted: GF VLI > EFLAGS: 00210097 (2.6.18.8-xen #2) > EIP is at evtchn_get_xen_pirq+0x35/0x40 > eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 > esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac > ds: 007b es: 007b ss: 0069 > Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 task.ti=ed384000) > Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 00000000 > 00000000 > 00000000 00000000 00000000 00000000 00000000 00000000 00000000 > 00000000 > 00000000 00000000 00000000 00000000 00000000 00000000 00000000 > 00000000 > Call Trace: > [<c0248aef>] startup_pirq+0x3f/0x250 > [<c0150b50>] setup_irq+0x160/0x1b0 > [<ee0dd270>] nfe_interrupt_handler+0x0/0x30 [nfemsg] > [<c0150c43>] request_irq+0xa3/0xc0 > [<ee02c8ad>] nfemsg_module_init+0x8ad/0x133e [nfemsg] > [<c030531b>] cond_resched+0x2b/0x40 > [<c0305369>] wait_for_completion+0x19/0xf0 > [<c0142818>] sys_init_module+0x148/0x1b50 > [<c010595f>] syscall_call+0x7/0xb > Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 44 c0 89 > d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 <0f> 0b > 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 > EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP 0069:ed385dac > ### card [0] start: host progs ### > -bash-3.2# > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: ------------[ cut here ]------------ > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: kernel BUG at >/usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:809> !> > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: invalid opcode: 0000 [#1] > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: SMP > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: CPU: 0 > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: EIP is at evtchn_get_xen_pirq+0x35/0x40 > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: ds: 007b es: 007b ss: 0069 > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 > task.ti=ed384000) > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 > 00000000 00000000 > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: 00000000 00000000 00000000 00000000 00000000 00000000 > 00000000 00000000 > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: 00000000 00000000 00000000 00000000 00000000 00000000 > 00000000 00000000 > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: Call Trace: > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 > 44 c0 89 d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 > <0f> 0b 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 > > Message from syslogd@drake at Sep 19 15:36:44 ... > kernel: EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP > 0069:ed385dac_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
So the question is what irq you pass in to request_irq() and where you got it from. Jan>>> Anish Bhatt <anish@gatech.edu> 23.09.08 07:04 >>>I put a printk before BUG_ON, Its expecting IRQT_PIRQ, but getting IRQT_UNBOUND instead. -Anish Jan Beulich wrote:>>>> Keir Fraser <keir.fraser@eu.citrix.com> 20.09.08 10:15 >>> >>>> >> On 3.3.0 you need to specify msi=1 on Xen''s command line to enable MSI. In >> xen-unstable, MSI support is always enabled. >> >> I changed a check in evtchn_get_xen_pirq() into a BUG_ON() as it looks to me >> like it should never trigger. Jan Beulich was the original authour of that >> function -- cc''ed in case he can indicate whether I''ve actually broken the >> function. :-) >> > > No, I think that BUG_ON() you added is valid. It definitely is in the context > of startup_pirq(). I''d be curious to know what the type of that irq really is, > but that cannot be determined from the backtrace alone. Anish, could you > perhaps add a simple printk() before the BUG_ON()? Unfortunately the > driver in question does not appear to be part of the tree, so we can''t even > check what it does prior to calling request_irq()... > > -- Keir > > On 19/9/08 20:53, "Anish Bhatt" <anish@cc.gatech.edu> wrote: > > >> lspci shows MSI enabled for PCI device. PCI passthrough works fine. >> However, as soon as the MSI driver for card is insmodded, kernel panics. >> This is on xen-unstable. Tried the same with xen-3.3.0 which is >> supposed to have MSI passthrough, but the same guest shows MSI as disabled. >> Any else seen this bug, or know of a workaround ? >> >> Trace is as follows : >> >> ------------[ cut here ]------------ >> kernel BUG at >> >> > /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:809> > ! > >> invalid opcode: 0000 [#1] >> SMP >> Modules linked in: nfemsg nfdvnet ipv6 binfmt_misc dm_mod nfe usbcore >> ext3 jbd processor fuse >> CPU: 0 >> EIP: 0061:[<c02487f5>] Tainted: GF VLI >> EFLAGS: 00210097 (2.6.18.8-xen #2) >> EIP is at evtchn_get_xen_pirq+0x35/0x40 >> eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >> esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >> ds: 007b es: 007b ss: 0069 >> Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 task.ti=ed384000) >> Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 00000000 >> 00000000 >> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >> 00000000 >> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >> 00000000 >> Call Trace: >> [<c0248aef>] startup_pirq+0x3f/0x250 >> [<c0150b50>] setup_irq+0x160/0x1b0 >> [<ee0dd270>] nfe_interrupt_handler+0x0/0x30 [nfemsg] >> [<c0150c43>] request_irq+0xa3/0xc0 >> [<ee02c8ad>] nfemsg_module_init+0x8ad/0x133e [nfemsg] >> [<c030531b>] cond_resched+0x2b/0x40 >> [<c0305369>] wait_for_completion+0x19/0xf0 >> [<c0142818>] sys_init_module+0x148/0x1b50 >> [<c010595f>] syscall_call+0x7/0xb >> Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 44 c0 89 >> d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 <0f> 0b >> 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >> EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP 0069:ed385dac >> ### card [0] start: host progs ### >> -bash-3.2# >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: ------------[ cut here ]------------ >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: kernel BUG at >> >> > /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:809> > ! > >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: invalid opcode: 0000 [#1] >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: SMP >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: CPU: 0 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: EIP is at evtchn_get_xen_pirq+0x35/0x40 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: ds: 007b es: 007b ss: 0069 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 >> task.ti=ed384000) >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 >> 00000000 00000000 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >> 00000000 00000000 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >> 00000000 00000000 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: Call Trace: >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 >> 44 c0 89 d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 >> <0f> 0b 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP >> 0069:ed385dac >> > > >-- As long as the music''s loud enough, we won''t hear the world falling apart. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Just some thoughts on Anish''s problem. It seems he reveals a bug in current code, I think. Currently, evnchn_map_pirq is hooked on msi_map_vector (pci_enable_msi->msi_capability_init->msi_map_vector). Here the pirq type is set to IRQT_PIRQ. Then request_irq should succeed since pirq type is already set. However, this path only works on the assumption that msi_map_vector and request_irq are executed in the same domain, to be more specific, dom0. For PV domU guest, there is no msi_map_vector in its pci_enable_msi ( PV domU actually asks dom0 to do the msi map, instead. This is a decision made by Keir and Yunhong long ago. I think it is base on security considerations). So its irq type is not set correctly (What''s more, it incorrectly sets the dom0''s irq type). Then request_irq can trigger this bug_on(). Anish, if you use the device in dom0 itself, does it also have this bug_on? Keir and Jan, Do you think this explanation is reasonable for issues Anish met? Best Regards Shan Haitao -----Original Message----- From: Jan Beulich [mailto:jbeulich@novell.com] Sent: 2008年9月23日 15:24 To: Anish Bhatt Cc: Keir Fraser; Shan, Haitao; xen-devel@lists.xensource.com Subject: Re: [Xen-devel] MSI causing softpanics in guest So the question is what irq you pass in to request_irq() and where you got it from. Jan>>> Anish Bhatt <anish@gatech.edu> 23.09.08 07:04 >>>I put a printk before BUG_ON, Its expecting IRQT_PIRQ, but getting IRQT_UNBOUND instead. -Anish Jan Beulich wrote:>>>> Keir Fraser <keir.fraser@eu.citrix.com> 20.09.08 10:15 >>> >>>> >> On 3.3.0 you need to specify msi=1 on Xen''s command line to enable MSI. In >> xen-unstable, MSI support is always enabled. >> >> I changed a check in evtchn_get_xen_pirq() into a BUG_ON() as it looks to me >> like it should never trigger. Jan Beulich was the original authour of that >> function -- cc''ed in case he can indicate whether I''ve actually broken the >> function. :-) >> > > No, I think that BUG_ON() you added is valid. It definitely is in the context > of startup_pirq(). I''d be curious to know what the type of that irq really is, > but that cannot be determined from the backtrace alone. Anish, could you > perhaps add a simple printk() before the BUG_ON()? Unfortunately the > driver in question does not appear to be part of the tree, so we can''t even > check what it does prior to calling request_irq()... > > -- Keir > > On 19/9/08 20:53, "Anish Bhatt" <anish@cc.gatech.edu> wrote: > > >> lspci shows MSI enabled for PCI device. PCI passthrough works fine. >> However, as soon as the MSI driver for card is insmodded, kernel panics. >> This is on xen-unstable. Tried the same with xen-3.3.0 which is >> supposed to have MSI passthrough, but the same guest shows MSI as disabled. >> Any else seen this bug, or know of a workaround ? >> >> Trace is as follows : >> >> ------------[ cut here ]------------ >> kernel BUG at >> >> > /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:809> > ! > >> invalid opcode: 0000 [#1] >> SMP >> Modules linked in: nfemsg nfdvnet ipv6 binfmt_misc dm_mod nfe usbcore >> ext3 jbd processor fuse >> CPU: 0 >> EIP: 0061:[<c02487f5>] Tainted: GF VLI >> EFLAGS: 00210097 (2.6.18.8-xen #2) >> EIP is at evtchn_get_xen_pirq+0x35/0x40 >> eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >> esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >> ds: 007b es: 007b ss: 0069 >> Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 task.ti=ed384000) >> Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 00000000 >> 00000000 >> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >> 00000000 >> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >> 00000000 >> Call Trace: >> [<c0248aef>] startup_pirq+0x3f/0x250 >> [<c0150b50>] setup_irq+0x160/0x1b0 >> [<ee0dd270>] nfe_interrupt_handler+0x0/0x30 [nfemsg] >> [<c0150c43>] request_irq+0xa3/0xc0 >> [<ee02c8ad>] nfemsg_module_init+0x8ad/0x133e [nfemsg] >> [<c030531b>] cond_resched+0x2b/0x40 >> [<c0305369>] wait_for_completion+0x19/0xf0 >> [<c0142818>] sys_init_module+0x148/0x1b50 >> [<c010595f>] syscall_call+0x7/0xb >> Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 44 c0 89 >> d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 <0f> 0b >> 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >> EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP 0069:ed385dac >> ### card [0] start: host progs ### >> -bash-3.2# >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: ------------[ cut here ]------------ >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: kernel BUG at >> >> > /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:809> > ! > >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: invalid opcode: 0000 [#1] >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: SMP >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: CPU: 0 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: EIP is at evtchn_get_xen_pirq+0x35/0x40 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: ds: 007b es: 007b ss: 0069 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 >> task.ti=ed384000) >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 >> 00000000 00000000 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >> 00000000 00000000 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >> 00000000 00000000 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: Call Trace: >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 >> 44 c0 89 d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 >> <0f> 0b 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >> >> Message from syslogd@drake at Sep 19 15:36:44 ... >> kernel: EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP >> 0069:ed385dac >> > > >-- As long as the music''s loud enough, we won''t hear the world falling apart. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 23/9/08 08:50, "Shan, Haitao" <haitao.shan@intel.com> wrote:> Just some thoughts on Anish''s problem. It seems he reveals a bug in current > code, I think. > > Currently, evnchn_map_pirq is hooked on msi_map_vector > (pci_enable_msi->msi_capability_init->msi_map_vector). Here the pirq type is > set to IRQT_PIRQ. Then request_irq should succeed since pirq type is already > set. > > However, this path only works on the assumption that msi_map_vector and > request_irq are executed in the same domain, to be more specific, dom0. > For PV domU guest, there is no msi_map_vector in its pci_enable_msi ( PV domU > actually asks dom0 to do the msi map, instead. This is a decision made by Keir > and Yunhong long ago. I think it is base on security considerations). So its > irq type is not set correctly (What''s more, it incorrectly sets the dom0''s irq > type). Then request_irq can trigger this bug_on(). > > Anish, if you use the device in dom0 itself, does it also have this bug_on? > > Keir and Jan, > Do you think this explanation is reasonable for issues Anish met?Yes, actually I remember thinking there should be a hook for Jan''s patch into pcifront, but then forgot about it. When pcifront gets back a pirq from pciback, it should evtchn_map_pirq(), right? -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Yes. I think so. But here is another problem, current code in dom0 returns irq number (irq number has meaning only in its domain, am I right?) not pirq number to PV domU guest. I am not sure whether this number can be used in evtchn_map_pirq in PV domU. Seems some modification is needed, I believe. Shan Haitao -----Original Message----- From: Keir Fraser [mailto:keir.fraser@eu.citrix.com] Sent: 2008年9月23日 16:15 To: Shan, Haitao; Jan Beulich; Anish Bhatt Cc: xen-devel@lists.xensource.com Subject: Re: [Xen-devel] MSI causing softpanics in guest On 23/9/08 08:50, "Shan, Haitao" <haitao.shan@intel.com> wrote:> Just some thoughts on Anish''s problem. It seems he reveals a bug in current > code, I think. > > Currently, evnchn_map_pirq is hooked on msi_map_vector > (pci_enable_msi->msi_capability_init->msi_map_vector). Here the pirq type is > set to IRQT_PIRQ. Then request_irq should succeed since pirq type is already > set. > > However, this path only works on the assumption that msi_map_vector and > request_irq are executed in the same domain, to be more specific, dom0. > For PV domU guest, there is no msi_map_vector in its pci_enable_msi ( PV domU > actually asks dom0 to do the msi map, instead. This is a decision made by Keir > and Yunhong long ago. I think it is base on security considerations). So its > irq type is not set correctly (What''s more, it incorrectly sets the dom0''s irq > type). Then request_irq can trigger this bug_on(). > > Anish, if you use the device in dom0 itself, does it also have this bug_on? > > Keir and Jan, > Do you think this explanation is reasonable for issues Anish met?Yes, actually I remember thinking there should be a hook for Jan''s patch into pcifront, but then forgot about it. When pcifront gets back a pirq from pciback, it should evtchn_map_pirq(), right? -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
pciback should be returning the number which the domU should present to Xen via EVTCHNOP_bind_pirq. Anything else would be meaningless. DomU then gets itself a Linux irq by ''irq = evtchn_map_pirq(-1, pirq_from_pciback)''. -- Keir On 23/9/08 09:26, "Shan, Haitao" <haitao.shan@intel.com> wrote:> Yes. I think so. > But here is another problem, current code in dom0 returns irq number (irq > number has meaning only in its domain, am I right?) not pirq number to PV domU > guest. I am not sure whether this number can be used in evtchn_map_pirq in PV > domU. Seems some modification is needed, I believe._______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Yes, exactly the same as what I am thinking. Shan Haitao -----Original Message----- From: Keir Fraser [mailto:keir.fraser@eu.citrix.com] Sent: 2008年9月23日 16:30 To: Shan, Haitao; Jan Beulich; Anish Bhatt Cc: xen-devel@lists.xensource.com Subject: Re: [Xen-devel] MSI causing softpanics in guest pciback should be returning the number which the domU should present to Xen via EVTCHNOP_bind_pirq. Anything else would be meaningless. DomU then gets itself a Linux irq by ''irq = evtchn_map_pirq(-1, pirq_from_pciback)''. -- Keir On 23/9/08 09:26, "Shan, Haitao" <haitao.shan@intel.com> wrote:> Yes. I think so. > But here is another problem, current code in dom0 returns irq number (irq > number has meaning only in its domain, am I right?) not pirq number to PV domU > guest. I am not sure whether this number can be used in evtchn_map_pirq in PV > domU. Seems some modification is needed, I believe._______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
No, the bug_on is not triggered in dom0. I also saw the following error message in xm dmesg, : /physdev.c:90:d0 remap pirq fa vector 49 while not unmap/ Its part of the code added for MSI, I wonder if its of any significance. -Anish Shan, Haitao wrote:> Just some thoughts on Anish''s problem. It seems he reveals a bug in current code, I think. > > Currently, evnchn_map_pirq is hooked on msi_map_vector (pci_enable_msi->msi_capability_init->msi_map_vector). Here the pirq type is set to IRQT_PIRQ. Then request_irq should succeed since pirq type is already set. > > However, this path only works on the assumption that msi_map_vector and request_irq are executed in the same domain, to be more specific, dom0. > For PV domU guest, there is no msi_map_vector in its pci_enable_msi ( PV domU actually asks dom0 to do the msi map, instead. This is a decision made by Keir and Yunhong long ago. I think it is base on security considerations). So its irq type is not set correctly (What''s more, it incorrectly sets the dom0''s irq type). Then request_irq can trigger this bug_on(). > > Anish, if you use the device in dom0 itself, does it also have this bug_on? > > Keir and Jan, > Do you think this explanation is reasonable for issues Anish met? > > Best Regards > Shan Haitao > > -----Original Message----- > From: Jan Beulich [mailto:jbeulich@novell.com] > Sent: 2008年9月23日 15:24 > To: Anish Bhatt > Cc: Keir Fraser; Shan, Haitao; xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] MSI causing softpanics in guest > > So the question is what irq you pass in to request_irq() and where you got > it from. Jan > > >>>> Anish Bhatt <anish@gatech.edu> 23.09.08 07:04 >>> >>>> > I put a printk before BUG_ON, Its expecting IRQT_PIRQ, but getting > IRQT_UNBOUND instead. > -Anish > > Jan Beulich wrote: > >>>>> Keir Fraser <keir.fraser@eu.citrix.com> 20.09.08 10:15 >>> >>>>> >>>>> >>> On 3.3.0 you need to specify msi=1 on Xen''s command line to enable MSI. In >>> xen-unstable, MSI support is always enabled. >>> >>> I changed a check in evtchn_get_xen_pirq() into a BUG_ON() as it looks to me >>> like it should never trigger. Jan Beulich was the original authour of that >>> function -- cc''ed in case he can indicate whether I''ve actually broken the >>> function. :-) >>> >>> >> No, I think that BUG_ON() you added is valid. It definitely is in the context >> of startup_pirq(). I''d be curious to know what the type of that irq really is, >> but that cannot be determined from the backtrace alone. Anish, could you >> perhaps add a simple printk() before the BUG_ON()? Unfortunately the >> driver in question does not appear to be part of the tree, so we can''t even >> check what it does prior to calling request_irq()... >> >> -- Keir >> >> On 19/9/08 20:53, "Anish Bhatt" <anish@cc.gatech.edu> wrote: >> >> >> >>> lspci shows MSI enabled for PCI device. PCI passthrough works fine. >>> However, as soon as the MSI driver for card is insmodded, kernel panics. >>> This is on xen-unstable. Tried the same with xen-3.3.0 which is >>> supposed to have MSI passthrough, but the same guest shows MSI as disabled. >>> Any else seen this bug, or know of a workaround ? >>> >>> Trace is as follows : >>> >>> ------------[ cut here ]------------ >>> kernel BUG at >>> >>> >>> >> /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:809> >> ! >> >> >>> invalid opcode: 0000 [#1] >>> SMP >>> Modules linked in: nfemsg nfdvnet ipv6 binfmt_misc dm_mod nfe usbcore >>> ext3 jbd processor fuse >>> CPU: 0 >>> EIP: 0061:[<c02487f5>] Tainted: GF VLI >>> EFLAGS: 00210097 (2.6.18.8-xen #2) >>> EIP is at evtchn_get_xen_pirq+0x35/0x40 >>> eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >>> esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >>> ds: 007b es: 007b ss: 0069 >>> Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 task.ti=ed384000) >>> Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 00000000 >>> 00000000 >>> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >>> 00000000 >>> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >>> 00000000 >>> Call Trace: >>> [<c0248aef>] startup_pirq+0x3f/0x250 >>> [<c0150b50>] setup_irq+0x160/0x1b0 >>> [<ee0dd270>] nfe_interrupt_handler+0x0/0x30 [nfemsg] >>> [<c0150c43>] request_irq+0xa3/0xc0 >>> [<ee02c8ad>] nfemsg_module_init+0x8ad/0x133e [nfemsg] >>> [<c030531b>] cond_resched+0x2b/0x40 >>> [<c0305369>] wait_for_completion+0x19/0xf0 >>> [<c0142818>] sys_init_module+0x148/0x1b50 >>> [<c010595f>] syscall_call+0x7/0xb >>> Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 44 c0 89 >>> d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 <0f> 0b >>> 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >>> EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP 0069:ed385dac >>> ### card [0] start: host progs ### >>> -bash-3.2# >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: ------------[ cut here ]------------ >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: kernel BUG at >>> >>> >>> >> /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:809> >> ! >> >> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: invalid opcode: 0000 [#1] >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: SMP >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: CPU: 0 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: EIP is at evtchn_get_xen_pirq+0x35/0x40 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: ds: 007b es: 007b ss: 0069 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 >>> task.ti=ed384000) >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 >>> 00000000 00000000 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >>> 00000000 00000000 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >>> 00000000 00000000 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: Call Trace: >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 >>> 44 c0 89 d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 >>> <0f> 0b 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP >>> 0069:ed385dac >>> >>> >> >> > > > -- > As long as the music''s loud enough, we won''t hear the world falling apart. > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >-- As long as the music''s loud enough, we won''t hear the world falling apart. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
It is supposed to work on dom0. This is a bug. Please refer to Keir''s reply in this mail thread. Shan Haitao -----Original Message----- From: Anish Bhatt [mailto:anish@cc.gatech.edu] Sent: 2008年9月24日 8:08 To: Shan, Haitao Cc: Jan Beulich; xen-devel@lists.xensource.com; Keir Fraser Subject: Re: [Xen-devel] MSI causing softpanics in guest No, the bug_on is not triggered in dom0. I also saw the following error message in xm dmesg, : /physdev.c:90:d0 remap pirq fa vector 49 while not unmap/ Its part of the code added for MSI, I wonder if its of any significance. -Anish Shan, Haitao wrote:> Just some thoughts on Anish''s problem. It seems he reveals a bug in current code, I think. > > Currently, evnchn_map_pirq is hooked on msi_map_vector (pci_enable_msi->msi_capability_init->msi_map_vector). Here the pirq type is set to IRQT_PIRQ. Then request_irq should succeed since pirq type is already set. > > However, this path only works on the assumption that msi_map_vector and request_irq are executed in the same domain, to be more specific, dom0. > For PV domU guest, there is no msi_map_vector in its pci_enable_msi ( PV domU actually asks dom0 to do the msi map, instead. This is a decision made by Keir and Yunhong long ago. I think it is base on security considerations). So its irq type is not set correctly (What''s more, it incorrectly sets the dom0''s irq type). Then request_irq can trigger this bug_on(). > > Anish, if you use the device in dom0 itself, does it also have this bug_on? > > Keir and Jan, > Do you think this explanation is reasonable for issues Anish met? > > Best Regards > Shan Haitao > > -----Original Message----- > From: Jan Beulich [mailto:jbeulich@novell.com] > Sent: 2008年9月23日 15:24 > To: Anish Bhatt > Cc: Keir Fraser; Shan, Haitao; xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] MSI causing softpanics in guest > > So the question is what irq you pass in to request_irq() and where you got > it from. Jan > > >>>> Anish Bhatt <anish@gatech.edu> 23.09.08 07:04 >>> >>>> > I put a printk before BUG_ON, Its expecting IRQT_PIRQ, but getting > IRQT_UNBOUND instead. > -Anish > > Jan Beulich wrote: > >>>>> Keir Fraser <keir.fraser@eu.citrix.com> 20.09.08 10:15 >>> >>>>> >>>>> >>> On 3.3.0 you need to specify msi=1 on Xen''s command line to enable MSI. In >>> xen-unstable, MSI support is always enabled. >>> >>> I changed a check in evtchn_get_xen_pirq() into a BUG_ON() as it looks to me >>> like it should never trigger. Jan Beulich was the original authour of that >>> function -- cc''ed in case he can indicate whether I''ve actually broken the >>> function. :-) >>> >>> >> No, I think that BUG_ON() you added is valid. It definitely is in the context >> of startup_pirq(). I''d be curious to know what the type of that irq really is, >> but that cannot be determined from the backtrace alone. Anish, could you >> perhaps add a simple printk() before the BUG_ON()? Unfortunately the >> driver in question does not appear to be part of the tree, so we can''t even >> check what it does prior to calling request_irq()... >> >> -- Keir >> >> On 19/9/08 20:53, "Anish Bhatt" <anish@cc.gatech.edu> wrote: >> >> >> >>> lspci shows MSI enabled for PCI device. PCI passthrough works fine. >>> However, as soon as the MSI driver for card is insmodded, kernel panics. >>> This is on xen-unstable. Tried the same with xen-3.3.0 which is >>> supposed to have MSI passthrough, but the same guest shows MSI as disabled. >>> Any else seen this bug, or know of a workaround ? >>> >>> Trace is as follows : >>> >>> ------------[ cut here ]------------ >>> kernel BUG at >>> >>> >>> >> /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:809> >> ! >> >> >>> invalid opcode: 0000 [#1] >>> SMP >>> Modules linked in: nfemsg nfdvnet ipv6 binfmt_misc dm_mod nfe usbcore >>> ext3 jbd processor fuse >>> CPU: 0 >>> EIP: 0061:[<c02487f5>] Tainted: GF VLI >>> EFLAGS: 00210097 (2.6.18.8-xen #2) >>> EIP is at evtchn_get_xen_pirq+0x35/0x40 >>> eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >>> esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >>> ds: 007b es: 007b ss: 0069 >>> Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 task.ti=ed384000) >>> Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 00000000 >>> 00000000 >>> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >>> 00000000 >>> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >>> 00000000 >>> Call Trace: >>> [<c0248aef>] startup_pirq+0x3f/0x250 >>> [<c0150b50>] setup_irq+0x160/0x1b0 >>> [<ee0dd270>] nfe_interrupt_handler+0x0/0x30 [nfemsg] >>> [<c0150c43>] request_irq+0xa3/0xc0 >>> [<ee02c8ad>] nfemsg_module_init+0x8ad/0x133e [nfemsg] >>> [<c030531b>] cond_resched+0x2b/0x40 >>> [<c0305369>] wait_for_completion+0x19/0xf0 >>> [<c0142818>] sys_init_module+0x148/0x1b50 >>> [<c010595f>] syscall_call+0x7/0xb >>> Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 44 c0 89 >>> d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 <0f> 0b >>> 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >>> EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP 0069:ed385dac >>> ### card [0] start: host progs ### >>> -bash-3.2# >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: ------------[ cut here ]------------ >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: kernel BUG at >>> >>> >>> >> /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:809> >> ! >> >> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: invalid opcode: 0000 [#1] >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: SMP >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: CPU: 0 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: EIP is at evtchn_get_xen_pirq+0x35/0x40 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: ds: 007b es: 007b ss: 0069 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 >>> task.ti=ed384000) >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 >>> 00000000 00000000 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >>> 00000000 00000000 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >>> 00000000 00000000 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: Call Trace: >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 >>> 44 c0 89 d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 >>> <0f> 0b 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >>> >>> Message from syslogd@drake at Sep 19 15:36:44 ... >>> kernel: EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP >>> 0069:ed385dac >>> >>> >> >> > > > -- > As long as the music''s loud enough, we won''t hear the world falling apart. > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >-- As long as the music''s loud enough, we won''t hear the world falling apart. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
So, which of us is going to patch pcifront? :-) -- Keir On 24/9/08 02:24, "Shan, Haitao" <haitao.shan@intel.com> wrote:> It is supposed to work on dom0. This is a bug. > Please refer to Keir''s reply in this mail thread. > > Shan Haitao > > -----Original Message----- > From: Anish Bhatt [mailto:anish@cc.gatech.edu] > Sent: 2008年9月24日 8:08 > To: Shan, Haitao > Cc: Jan Beulich; xen-devel@lists.xensource.com; Keir Fraser > Subject: Re: [Xen-devel] MSI causing softpanics in guest > > No, the bug_on is not triggered in dom0. I also saw the following error > message in xm dmesg, : > /physdev.c:90:d0 remap pirq fa vector 49 while not unmap/ > Its part of the code added for MSI, I wonder if its of any significance. > > -Anish > > Shan, Haitao wrote: >> Just some thoughts on Anish''s problem. It seems he reveals a bug in current >> code, I think. >> >> Currently, evnchn_map_pirq is hooked on msi_map_vector >> (pci_enable_msi->msi_capability_init->msi_map_vector). Here the pirq type is >> set to IRQT_PIRQ. Then request_irq should succeed since pirq type is already >> set. >> >> However, this path only works on the assumption that msi_map_vector and >> request_irq are executed in the same domain, to be more specific, dom0. >> For PV domU guest, there is no msi_map_vector in its pci_enable_msi ( PV domU >> actually asks dom0 to do the msi map, instead. This is a decision made by >> Keir and Yunhong long ago. I think it is base on security considerations). So >> its irq type is not set correctly (What''s more, it incorrectly sets the >> dom0''s irq type). Then request_irq can trigger this bug_on(). >> >> Anish, if you use the device in dom0 itself, does it also have this bug_on? >> >> Keir and Jan, >> Do you think this explanation is reasonable for issues Anish met? >> >> Best Regards >> Shan Haitao >> >> -----Original Message----- >> From: Jan Beulich [mailto:jbeulich@novell.com] >> Sent: 2008年9月23日 15:24 >> To: Anish Bhatt >> Cc: Keir Fraser; Shan, Haitao; xen-devel@lists.xensource.com >> Subject: Re: [Xen-devel] MSI causing softpanics in guest >> >> So the question is what irq you pass in to request_irq() and where you got >> it from. Jan >> >> >>>>> Anish Bhatt <anish@gatech.edu> 23.09.08 07:04 >>> >>>>> >> I put a printk before BUG_ON, Its expecting IRQT_PIRQ, but getting >> IRQT_UNBOUND instead. >> -Anish >> >> Jan Beulich wrote: >> >>>>>> Keir Fraser <keir.fraser@eu.citrix.com> 20.09.08 10:15 >>> >>>>>> >>>>>> >>>> On 3.3.0 you need to specify msi=1 on Xen''s command line to enable MSI. In >>>> xen-unstable, MSI support is always enabled. >>>> >>>> I changed a check in evtchn_get_xen_pirq() into a BUG_ON() as it looks to >>>> me >>>> like it should never trigger. Jan Beulich was the original authour of that >>>> function -- cc''ed in case he can indicate whether I''ve actually broken the >>>> function. :-) >>>> >>>> >>> No, I think that BUG_ON() you added is valid. It definitely is in the >>> context >>> of startup_pirq(). I''d be curious to know what the type of that irq really >>> is, >>> but that cannot be determined from the backtrace alone. Anish, could you >>> perhaps add a simple printk() before the BUG_ON()? Unfortunately the >>> driver in question does not appear to be part of the tree, so we can''t even >>> check what it does prior to calling request_irq()... >>> >>> -- Keir >>> >>> On 19/9/08 20:53, "Anish Bhatt" <anish@cc.gatech.edu> wrote: >>> >>> >>> >>>> lspci shows MSI enabled for PCI device. PCI passthrough works fine. >>>> However, as soon as the MSI driver for card is insmodded, kernel panics. >>>> This is on xen-unstable. Tried the same with xen-3.3.0 which is >>>> supposed to have MSI passthrough, but the same guest shows MSI as disabled. >>>> Any else seen this bug, or know of a workaround ? >>>> >>>> Trace is as follows : >>>> >>>> ------------[ cut here ]------------ >>>> kernel BUG at >>>> >>>> >>>> >>> /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:8 >>> 09> >>> ! >>> >>> >>>> invalid opcode: 0000 [#1] >>>> SMP >>>> Modules linked in: nfemsg nfdvnet ipv6 binfmt_misc dm_mod nfe usbcore >>>> ext3 jbd processor fuse >>>> CPU: 0 >>>> EIP: 0061:[<c02487f5>] Tainted: GF VLI >>>> EFLAGS: 00210097 (2.6.18.8-xen #2) >>>> EIP is at evtchn_get_xen_pirq+0x35/0x40 >>>> eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >>>> esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >>>> ds: 007b es: 007b ss: 0069 >>>> Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 task.ti=ed384000) >>>> Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 >>>> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 >>>> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 >>>> Call Trace: >>>> [<c0248aef>] startup_pirq+0x3f/0x250 >>>> [<c0150b50>] setup_irq+0x160/0x1b0 >>>> [<ee0dd270>] nfe_interrupt_handler+0x0/0x30 [nfemsg] >>>> [<c0150c43>] request_irq+0xa3/0xc0 >>>> [<ee02c8ad>] nfemsg_module_init+0x8ad/0x133e [nfemsg] >>>> [<c030531b>] cond_resched+0x2b/0x40 >>>> [<c0305369>] wait_for_completion+0x19/0xf0 >>>> [<c0142818>] sys_init_module+0x148/0x1b50 >>>> [<c010595f>] syscall_call+0x7/0xb >>>> Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 44 c0 89 >>>> d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 <0f> 0b >>>> 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >>>> EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP 0069:ed385dac >>>> ### card [0] start: host progs ### >>>> -bash-3.2# >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: ------------[ cut here ]------------ >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: kernel BUG at >>>> >>>> >>>> >>> /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:8 >>> 09> >>> ! >>> >>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: invalid opcode: 0000 [#1] >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: SMP >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: CPU: 0 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: EIP is at evtchn_get_xen_pirq+0x35/0x40 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: ds: 007b es: 007b ss: 0069 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 >>>> task.ti=ed384000) >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 00000000 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 00000000 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 00000000 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: Call Trace: >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 >>>> 44 c0 89 d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 >>>> <0f> 0b 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP >>>> 0069:ed385dac >>>> >>>> >>> >>> >> >> >> -- >> As long as the music''s loud enough, we won''t hear the world falling apart. >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> > > > -- > As long as the music''s loud enough, we won''t hear the world falling apart. >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
OK. Let me take this AR. :) I will cook a patch for that. Shan Haitao -----Original Message----- From: Keir Fraser [mailto:keir.fraser@eu.citrix.com] Sent: 2008年9月24日 14:05 To: Shan, Haitao; Anish Bhatt Cc: Jan Beulich; xen-devel@lists.xensource.com Subject: Re: [Xen-devel] MSI causing softpanics in guest So, which of us is going to patch pcifront? :-) -- Keir On 24/9/08 02:24, "Shan, Haitao" <haitao.shan@intel.com> wrote:> It is supposed to work on dom0. This is a bug. > Please refer to Keir''s reply in this mail thread. > > Shan Haitao > > -----Original Message----- > From: Anish Bhatt [mailto:anish@cc.gatech.edu] > Sent: 2008年9月24日 8:08 > To: Shan, Haitao > Cc: Jan Beulich; xen-devel@lists.xensource.com; Keir Fraser > Subject: Re: [Xen-devel] MSI causing softpanics in guest > > No, the bug_on is not triggered in dom0. I also saw the following error > message in xm dmesg, : > /physdev.c:90:d0 remap pirq fa vector 49 while not unmap/ > Its part of the code added for MSI, I wonder if its of any significance. > > -Anish > > Shan, Haitao wrote: >> Just some thoughts on Anish''s problem. It seems he reveals a bug in current >> code, I think. >> >> Currently, evnchn_map_pirq is hooked on msi_map_vector >> (pci_enable_msi->msi_capability_init->msi_map_vector). Here the pirq type is >> set to IRQT_PIRQ. Then request_irq should succeed since pirq type is already >> set. >> >> However, this path only works on the assumption that msi_map_vector and >> request_irq are executed in the same domain, to be more specific, dom0. >> For PV domU guest, there is no msi_map_vector in its pci_enable_msi ( PV domU >> actually asks dom0 to do the msi map, instead. This is a decision made by >> Keir and Yunhong long ago. I think it is base on security considerations). So >> its irq type is not set correctly (What''s more, it incorrectly sets the >> dom0''s irq type). Then request_irq can trigger this bug_on(). >> >> Anish, if you use the device in dom0 itself, does it also have this bug_on? >> >> Keir and Jan, >> Do you think this explanation is reasonable for issues Anish met? >> >> Best Regards >> Shan Haitao >> >> -----Original Message----- >> From: Jan Beulich [mailto:jbeulich@novell.com] >> Sent: 2008年9月23日 15:24 >> To: Anish Bhatt >> Cc: Keir Fraser; Shan, Haitao; xen-devel@lists.xensource.com >> Subject: Re: [Xen-devel] MSI causing softpanics in guest >> >> So the question is what irq you pass in to request_irq() and where you got >> it from. Jan >> >> >>>>> Anish Bhatt <anish@gatech.edu> 23.09.08 07:04 >>> >>>>> >> I put a printk before BUG_ON, Its expecting IRQT_PIRQ, but getting >> IRQT_UNBOUND instead. >> -Anish >> >> Jan Beulich wrote: >> >>>>>> Keir Fraser <keir.fraser@eu.citrix.com> 20.09.08 10:15 >>> >>>>>> >>>>>> >>>> On 3.3.0 you need to specify msi=1 on Xen''s command line to enable MSI. In >>>> xen-unstable, MSI support is always enabled. >>>> >>>> I changed a check in evtchn_get_xen_pirq() into a BUG_ON() as it looks to >>>> me >>>> like it should never trigger. Jan Beulich was the original authour of that >>>> function -- cc''ed in case he can indicate whether I''ve actually broken the >>>> function. :-) >>>> >>>> >>> No, I think that BUG_ON() you added is valid. It definitely is in the >>> context >>> of startup_pirq(). I''d be curious to know what the type of that irq really >>> is, >>> but that cannot be determined from the backtrace alone. Anish, could you >>> perhaps add a simple printk() before the BUG_ON()? Unfortunately the >>> driver in question does not appear to be part of the tree, so we can''t even >>> check what it does prior to calling request_irq()... >>> >>> -- Keir >>> >>> On 19/9/08 20:53, "Anish Bhatt" <anish@cc.gatech.edu> wrote: >>> >>> >>> >>>> lspci shows MSI enabled for PCI device. PCI passthrough works fine. >>>> However, as soon as the MSI driver for card is insmodded, kernel panics. >>>> This is on xen-unstable. Tried the same with xen-3.3.0 which is >>>> supposed to have MSI passthrough, but the same guest shows MSI as disabled. >>>> Any else seen this bug, or know of a workaround ? >>>> >>>> Trace is as follows : >>>> >>>> ------------[ cut here ]------------ >>>> kernel BUG at >>>> >>>> >>>> >>> /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:8 >>> 09> >>> ! >>> >>> >>>> invalid opcode: 0000 [#1] >>>> SMP >>>> Modules linked in: nfemsg nfdvnet ipv6 binfmt_misc dm_mod nfe usbcore >>>> ext3 jbd processor fuse >>>> CPU: 0 >>>> EIP: 0061:[<c02487f5>] Tainted: GF VLI >>>> EFLAGS: 00210097 (2.6.18.8-xen #2) >>>> EIP is at evtchn_get_xen_pirq+0x35/0x40 >>>> eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >>>> esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >>>> ds: 007b es: 007b ss: 0069 >>>> Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 task.ti=ed384000) >>>> Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 >>>> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 >>>> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 >>>> Call Trace: >>>> [<c0248aef>] startup_pirq+0x3f/0x250 >>>> [<c0150b50>] setup_irq+0x160/0x1b0 >>>> [<ee0dd270>] nfe_interrupt_handler+0x0/0x30 [nfemsg] >>>> [<c0150c43>] request_irq+0xa3/0xc0 >>>> [<ee02c8ad>] nfemsg_module_init+0x8ad/0x133e [nfemsg] >>>> [<c030531b>] cond_resched+0x2b/0x40 >>>> [<c0305369>] wait_for_completion+0x19/0xf0 >>>> [<c0142818>] sys_init_module+0x148/0x1b50 >>>> [<c010595f>] syscall_call+0x7/0xb >>>> Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 44 c0 89 >>>> d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 <0f> 0b >>>> 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >>>> EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP 0069:ed385dac >>>> ### card [0] start: host progs ### >>>> -bash-3.2# >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: ------------[ cut here ]------------ >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: kernel BUG at >>>> >>>> >>>> >>> /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:8 >>> 09> >>> ! >>> >>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: invalid opcode: 0000 [#1] >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: SMP >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: CPU: 0 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: EIP is at evtchn_get_xen_pirq+0x35/0x40 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: ds: 007b es: 007b ss: 0069 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 >>>> task.ti=ed384000) >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 00000000 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 00000000 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 00000000 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: Call Trace: >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 >>>> 44 c0 89 d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 >>>> <0f> 0b 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP >>>> 0069:ed385dac >>>> >>>> >>> >>> >> >> >> -- >> As long as the music''s loud enough, we won''t hear the world falling apart. >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> > > > -- > As long as the music''s loud enough, we won''t hear the world falling apart. >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi, Keir, This is an RFC patch, which is not tested now (It will cost some time to setup my test environments). I want to ensure I am on the same page with you. So could you pleas have a look and point to me which part I may be missing? Thanks! Shan Haitao -----Original Message----- From: Keir Fraser [mailto:keir.fraser@eu.citrix.com] Sent: 2008年9月24日 14:05 To: Shan, Haitao; Anish Bhatt Cc: Jan Beulich; xen-devel@lists.xensource.com Subject: Re: [Xen-devel] MSI causing softpanics in guest So, which of us is going to patch pcifront? :-) -- Keir On 24/9/08 02:24, "Shan, Haitao" <haitao.shan@intel.com> wrote:> It is supposed to work on dom0. This is a bug. > Please refer to Keir''s reply in this mail thread. > > Shan Haitao > > -----Original Message----- > From: Anish Bhatt [mailto:anish@cc.gatech.edu] > Sent: 2008年9月24日 8:08 > To: Shan, Haitao > Cc: Jan Beulich; xen-devel@lists.xensource.com; Keir Fraser > Subject: Re: [Xen-devel] MSI causing softpanics in guest > > No, the bug_on is not triggered in dom0. I also saw the following error > message in xm dmesg, : > /physdev.c:90:d0 remap pirq fa vector 49 while not unmap/ > Its part of the code added for MSI, I wonder if its of any significance. > > -Anish > > Shan, Haitao wrote: >> Just some thoughts on Anish''s problem. It seems he reveals a bug in current >> code, I think. >> >> Currently, evnchn_map_pirq is hooked on msi_map_vector >> (pci_enable_msi->msi_capability_init->msi_map_vector). Here the pirq type is >> set to IRQT_PIRQ. Then request_irq should succeed since pirq type is already >> set. >> >> However, this path only works on the assumption that msi_map_vector and >> request_irq are executed in the same domain, to be more specific, dom0. >> For PV domU guest, there is no msi_map_vector in its pci_enable_msi ( PV domU >> actually asks dom0 to do the msi map, instead. This is a decision made by >> Keir and Yunhong long ago. I think it is base on security considerations). So >> its irq type is not set correctly (What''s more, it incorrectly sets the >> dom0''s irq type). Then request_irq can trigger this bug_on(). >> >> Anish, if you use the device in dom0 itself, does it also have this bug_on? >> >> Keir and Jan, >> Do you think this explanation is reasonable for issues Anish met? >> >> Best Regards >> Shan Haitao >> >> -----Original Message----- >> From: Jan Beulich [mailto:jbeulich@novell.com] >> Sent: 2008年9月23日 15:24 >> To: Anish Bhatt >> Cc: Keir Fraser; Shan, Haitao; xen-devel@lists.xensource.com >> Subject: Re: [Xen-devel] MSI causing softpanics in guest >> >> So the question is what irq you pass in to request_irq() and where you got >> it from. Jan >> >> >>>>> Anish Bhatt <anish@gatech.edu> 23.09.08 07:04 >>> >>>>> >> I put a printk before BUG_ON, Its expecting IRQT_PIRQ, but getting >> IRQT_UNBOUND instead. >> -Anish >> >> Jan Beulich wrote: >> >>>>>> Keir Fraser <keir.fraser@eu.citrix.com> 20.09.08 10:15 >>> >>>>>> >>>>>> >>>> On 3.3.0 you need to specify msi=1 on Xen''s command line to enable MSI. In >>>> xen-unstable, MSI support is always enabled. >>>> >>>> I changed a check in evtchn_get_xen_pirq() into a BUG_ON() as it looks to >>>> me >>>> like it should never trigger. Jan Beulich was the original authour of that >>>> function -- cc''ed in case he can indicate whether I''ve actually broken the >>>> function. :-) >>>> >>>> >>> No, I think that BUG_ON() you added is valid. It definitely is in the >>> context >>> of startup_pirq(). I''d be curious to know what the type of that irq really >>> is, >>> but that cannot be determined from the backtrace alone. Anish, could you >>> perhaps add a simple printk() before the BUG_ON()? Unfortunately the >>> driver in question does not appear to be part of the tree, so we can''t even >>> check what it does prior to calling request_irq()... >>> >>> -- Keir >>> >>> On 19/9/08 20:53, "Anish Bhatt" <anish@cc.gatech.edu> wrote: >>> >>> >>> >>>> lspci shows MSI enabled for PCI device. PCI passthrough works fine. >>>> However, as soon as the MSI driver for card is insmodded, kernel panics. >>>> This is on xen-unstable. Tried the same with xen-3.3.0 which is >>>> supposed to have MSI passthrough, but the same guest shows MSI as disabled. >>>> Any else seen this bug, or know of a workaround ? >>>> >>>> Trace is as follows : >>>> >>>> ------------[ cut here ]------------ >>>> kernel BUG at >>>> >>>> >>>> >>> /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:8 >>> 09> >>> ! >>> >>> >>>> invalid opcode: 0000 [#1] >>>> SMP >>>> Modules linked in: nfemsg nfdvnet ipv6 binfmt_misc dm_mod nfe usbcore >>>> ext3 jbd processor fuse >>>> CPU: 0 >>>> EIP: 0061:[<c02487f5>] Tainted: GF VLI >>>> EFLAGS: 00210097 (2.6.18.8-xen #2) >>>> EIP is at evtchn_get_xen_pirq+0x35/0x40 >>>> eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >>>> esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >>>> ds: 007b es: 007b ss: 0069 >>>> Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 task.ti=ed384000) >>>> Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 >>>> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 >>>> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 >>>> Call Trace: >>>> [<c0248aef>] startup_pirq+0x3f/0x250 >>>> [<c0150b50>] setup_irq+0x160/0x1b0 >>>> [<ee0dd270>] nfe_interrupt_handler+0x0/0x30 [nfemsg] >>>> [<c0150c43>] request_irq+0xa3/0xc0 >>>> [<ee02c8ad>] nfemsg_module_init+0x8ad/0x133e [nfemsg] >>>> [<c030531b>] cond_resched+0x2b/0x40 >>>> [<c0305369>] wait_for_completion+0x19/0xf0 >>>> [<c0142818>] sys_init_module+0x148/0x1b50 >>>> [<c010595f>] syscall_call+0x7/0xb >>>> Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 44 c0 89 >>>> d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 <0f> 0b >>>> 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >>>> EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP 0069:ed385dac >>>> ### card [0] start: host progs ### >>>> -bash-3.2# >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: ------------[ cut here ]------------ >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: kernel BUG at >>>> >>>> >>>> >>> /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:8 >>> 09> >>> ! >>> >>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: invalid opcode: 0000 [#1] >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: SMP >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: CPU: 0 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: EIP is at evtchn_get_xen_pirq+0x35/0x40 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: ds: 007b es: 007b ss: 0069 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 >>>> task.ti=ed384000) >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 00000000 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 00000000 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 00000000 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: Call Trace: >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 >>>> 44 c0 89 d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 >>>> <0f> 0b 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP >>>> 0069:ed385dac >>>> >>>> >>> >>> >> >> >> -- >> As long as the music''s loud enough, we won''t hear the world falling apart. >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> > > > -- > As long as the music''s loud enough, we won''t hear the world falling apart. >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Yes, it''s a better job than I could have managed. -- Keir On 24/9/08 10:38, "Shan, Haitao" <haitao.shan@intel.com> wrote:> Hi, Keir, > > This is an RFC patch, which is not tested now (It will cost some time to setup > my test environments). I want to ensure I am on the same page with you. So > could you pleas have a look and point to me which part I may be missing? > Thanks! > > Shan Haitao > > -----Original Message----- > From: Keir Fraser [mailto:keir.fraser@eu.citrix.com] > Sent: 2008年9月24日 14:05 > To: Shan, Haitao; Anish Bhatt > Cc: Jan Beulich; xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] MSI causing softpanics in guest > > So, which of us is going to patch pcifront? :-) > > -- Keir > > On 24/9/08 02:24, "Shan, Haitao" <haitao.shan@intel.com> wrote: > >> It is supposed to work on dom0. This is a bug. >> Please refer to Keir''s reply in this mail thread. >> >> Shan Haitao >> >> -----Original Message----- >> From: Anish Bhatt [mailto:anish@cc.gatech.edu] >> Sent: 2008年9月24日 8:08 >> To: Shan, Haitao >> Cc: Jan Beulich; xen-devel@lists.xensource.com; Keir Fraser >> Subject: Re: [Xen-devel] MSI causing softpanics in guest >> >> No, the bug_on is not triggered in dom0. I also saw the following error >> message in xm dmesg, : >> /physdev.c:90:d0 remap pirq fa vector 49 while not unmap/ >> Its part of the code added for MSI, I wonder if its of any significance. >> >> -Anish >> >> Shan, Haitao wrote: >>> Just some thoughts on Anish''s problem. It seems he reveals a bug in current >>> code, I think. >>> >>> Currently, evnchn_map_pirq is hooked on msi_map_vector >>> (pci_enable_msi->msi_capability_init->msi_map_vector). Here the pirq type is >>> set to IRQT_PIRQ. Then request_irq should succeed since pirq type is already >>> set. >>> >>> However, this path only works on the assumption that msi_map_vector and >>> request_irq are executed in the same domain, to be more specific, dom0. >>> For PV domU guest, there is no msi_map_vector in its pci_enable_msi ( PV >>> domU >>> actually asks dom0 to do the msi map, instead. This is a decision made by >>> Keir and Yunhong long ago. I think it is base on security considerations). >>> So >>> its irq type is not set correctly (What''s more, it incorrectly sets the >>> dom0''s irq type). Then request_irq can trigger this bug_on(). >>> >>> Anish, if you use the device in dom0 itself, does it also have this bug_on? >>> >>> Keir and Jan, >>> Do you think this explanation is reasonable for issues Anish met? >>> >>> Best Regards >>> Shan Haitao >>> >>> -----Original Message----- >>> From: Jan Beulich [mailto:jbeulich@novell.com] >>> Sent: 2008年9月23日 15:24 >>> To: Anish Bhatt >>> Cc: Keir Fraser; Shan, Haitao; xen-devel@lists.xensource.com >>> Subject: Re: [Xen-devel] MSI causing softpanics in guest >>> >>> So the question is what irq you pass in to request_irq() and where you got >>> it from. Jan >>> >>> >>>>>> Anish Bhatt <anish@gatech.edu> 23.09.08 07:04 >>> >>>>>> >>> I put a printk before BUG_ON, Its expecting IRQT_PIRQ, but getting >>> IRQT_UNBOUND instead. >>> -Anish >>> >>> Jan Beulich wrote: >>> >>>>>>> Keir Fraser <keir.fraser@eu.citrix.com> 20.09.08 10:15 >>> >>>>>>> >>>>>>> >>>>> On 3.3.0 you need to specify msi=1 on Xen''s command line to enable MSI. In >>>>> xen-unstable, MSI support is always enabled. >>>>> >>>>> I changed a check in evtchn_get_xen_pirq() into a BUG_ON() as it looks to >>>>> me >>>>> like it should never trigger. Jan Beulich was the original authour of that >>>>> function -- cc''ed in case he can indicate whether I''ve actually broken the >>>>> function. :-) >>>>> >>>>> >>>> No, I think that BUG_ON() you added is valid. It definitely is in the >>>> context >>>> of startup_pirq(). I''d be curious to know what the type of that irq really >>>> is, >>>> but that cannot be determined from the backtrace alone. Anish, could you >>>> perhaps add a simple printk() before the BUG_ON()? Unfortunately the >>>> driver in question does not appear to be part of the tree, so we can''t even >>>> check what it does prior to calling request_irq()... >>>> >>>> -- Keir >>>> >>>> On 19/9/08 20:53, "Anish Bhatt" <anish@cc.gatech.edu> wrote: >>>> >>>> >>>> >>>>> lspci shows MSI enabled for PCI device. PCI passthrough works fine. >>>>> However, as soon as the MSI driver for card is insmodded, kernel panics. >>>>> This is on xen-unstable. Tried the same with xen-3.3.0 which is >>>>> supposed to have MSI passthrough, but the same guest shows MSI as >>>>> disabled. >>>>> Any else seen this bug, or know of a workaround ? >>>>> >>>>> Trace is as follows : >>>>> >>>>> ------------[ cut here ]------------ >>>>> kernel BUG at >>>>> >>>>> >>>>> >>>>/usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:>>>> 8>>>> 09> >>>> ! >>>> >>>> >>>>> invalid opcode: 0000 [#1] >>>>> SMP >>>>> Modules linked in: nfemsg nfdvnet ipv6 binfmt_misc dm_mod nfe usbcore >>>>> ext3 jbd processor fuse >>>>> CPU: 0 >>>>> EIP: 0061:[<c02487f5>] Tainted: GF VLI >>>>> EFLAGS: 00210097 (2.6.18.8-xen #2) >>>>> EIP is at evtchn_get_xen_pirq+0x35/0x40 >>>>> eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >>>>> esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >>>>> ds: 007b es: 007b ss: 0069 >>>>> Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 task.ti=ed384000) >>>>> Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 00000000 >>>>> 00000000 >>>>> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >>>>> 00000000 >>>>> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >>>>> 00000000 >>>>> Call Trace: >>>>> [<c0248aef>] startup_pirq+0x3f/0x250 >>>>> [<c0150b50>] setup_irq+0x160/0x1b0 >>>>> [<ee0dd270>] nfe_interrupt_handler+0x0/0x30 [nfemsg] >>>>> [<c0150c43>] request_irq+0xa3/0xc0 >>>>> [<ee02c8ad>] nfemsg_module_init+0x8ad/0x133e [nfemsg] >>>>> [<c030531b>] cond_resched+0x2b/0x40 >>>>> [<c0305369>] wait_for_completion+0x19/0xf0 >>>>> [<c0142818>] sys_init_module+0x148/0x1b50 >>>>> [<c010595f>] syscall_call+0x7/0xb >>>>> Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 44 c0 89 >>>>> d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 <0f> 0b >>>>> 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >>>>> EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP 0069:ed385dac >>>>> ### card [0] start: host progs ### >>>>> -bash-3.2# >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: ------------[ cut here ]------------ >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: kernel BUG at >>>>> >>>>> >>>>> >>>>/usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:>>>> 8>>>> 09> >>>> ! >>>> >>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: invalid opcode: 0000 [#1] >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: SMP >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: CPU: 0 >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: EIP is at evtchn_get_xen_pirq+0x35/0x40 >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: ds: 007b es: 007b ss: 0069 >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 >>>>> task.ti=ed384000) >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 >>>>> 00000000 00000000 >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >>>>> 00000000 00000000 >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >>>>> 00000000 00000000 >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: Call Trace: >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 >>>>> 44 c0 89 d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 >>>>> <0f> 0b 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP >>>>> 0069:ed385dac >>>>> >>>>> >>>> >>>> >>> >>> >>> -- >>> As long as the music''s loud enough, we won''t hear the world falling apart. >>> >>> >>> _______________________________________________ >>> Xen-devel mailing list >>> Xen-devel@lists.xensource.com >>> http://lists.xensource.com/xen-devel >>> >> >> >> -- >> As long as the music''s loud enough, we won''t hear the world falling apart. >> > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>--- a/drivers/pci/msi-xen.c Mon Sep 22 16:08:10 2008 +0100 >+++ b/drivers/pci/msi-xen.c Wed Sep 24 01:45:16 2008 +0800 >@@ -158,7 +158,11 @@ static int msi_unmap_pirq(struct pci_dev > int rc; > > unmap.domid = msi_get_dev_owner(dev); >- unmap.pirq = evtchn_get_xen_pirq(pirq); >+ /* See comments in msi_map_pirq_to_vector, input parameter pirq >+ * mean irq number only if the device belongs to dom0 itself. >+ */ >+ if (unmap.domid == DOMID_SELF) >+ unmap.pirq = evtchn_get_xen_pirq(pirq);This seems to leave unmap.pirq uninitialized in the !DOMID_SELF case. Otherwise, seems a reasonable alternative to doing it in pcifront/pciback. Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Works fine now with your patch, but I still see the '' physdev.c:90:d0 remap pirq fa vector 49 while not unmap'' message in dom0 xm dmesg. Is that something to be bothered about ? -Anish Shan, Haitao wrote:> It is supposed to work on dom0. This is a bug. > Please refer to Keir''s reply in this mail thread. > > Shan Haitao > > -----Original Message----- > From: Anish Bhatt [mailto:anish@cc.gatech.edu] > Sent: 2008年9月24日 8:08 > To: Shan, Haitao > Cc: Jan Beulich; xen-devel@lists.xensource.com; Keir Fraser > Subject: Re: [Xen-devel] MSI causing softpanics in guest > > No, the bug_on is not triggered in dom0. I also saw the following error > message in xm dmesg, : > /physdev.c:90:d0 remap pirq fa vector 49 while not unmap/ > Its part of the code added for MSI, I wonder if its of any significance. > > -Anish > > Shan, Haitao wrote: > >> Just some thoughts on Anish''s problem. It seems he reveals a bug in current code, I think. >> >> Currently, evnchn_map_pirq is hooked on msi_map_vector (pci_enable_msi->msi_capability_init->msi_map_vector). Here the pirq type is set to IRQT_PIRQ. Then request_irq should succeed since pirq type is already set. >> >> However, this path only works on the assumption that msi_map_vector and request_irq are executed in the same domain, to be more specific, dom0. >> For PV domU guest, there is no msi_map_vector in its pci_enable_msi ( PV domU actually asks dom0 to do the msi map, instead. This is a decision made by Keir and Yunhong long ago. I think it is base on security considerations). So its irq type is not set correctly (What''s more, it incorrectly sets the dom0''s irq type). Then request_irq can trigger this bug_on(). >> >> Anish, if you use the device in dom0 itself, does it also have this bug_on? >> >> Keir and Jan, >> Do you think this explanation is reasonable for issues Anish met? >> >> Best Regards >> Shan Haitao >> >> -----Original Message----- >> From: Jan Beulich [mailto:jbeulich@novell.com] >> Sent: 2008年9月23日 15:24 >> To: Anish Bhatt >> Cc: Keir Fraser; Shan, Haitao; xen-devel@lists.xensource.com >> Subject: Re: [Xen-devel] MSI causing softpanics in guest >> >> So the question is what irq you pass in to request_irq() and where you got >> it from. Jan >> >> >> >>>>> Anish Bhatt <anish@gatech.edu> 23.09.08 07:04 >>> >>>>> >>>>> >> I put a printk before BUG_ON, Its expecting IRQT_PIRQ, but getting >> IRQT_UNBOUND instead. >> -Anish >> >> Jan Beulich wrote: >> >> >>>>>> Keir Fraser <keir.fraser@eu.citrix.com> 20.09.08 10:15 >>> >>>>>> >>>>>> >>>>>> >>>> On 3.3.0 you need to specify msi=1 on Xen''s command line to enable MSI. In >>>> xen-unstable, MSI support is always enabled. >>>> >>>> I changed a check in evtchn_get_xen_pirq() into a BUG_ON() as it looks to me >>>> like it should never trigger. Jan Beulich was the original authour of that >>>> function -- cc''ed in case he can indicate whether I''ve actually broken the >>>> function. :-) >>>> >>>> >>>> >>> No, I think that BUG_ON() you added is valid. It definitely is in the context >>> of startup_pirq(). I''d be curious to know what the type of that irq really is, >>> but that cannot be determined from the backtrace alone. Anish, could you >>> perhaps add a simple printk() before the BUG_ON()? Unfortunately the >>> driver in question does not appear to be part of the tree, so we can''t even >>> check what it does prior to calling request_irq()... >>> >>> -- Keir >>> >>> On 19/9/08 20:53, "Anish Bhatt" <anish@cc.gatech.edu> wrote: >>> >>> >>> >>> >>>> lspci shows MSI enabled for PCI device. PCI passthrough works fine. >>>> However, as soon as the MSI driver for card is insmodded, kernel panics. >>>> This is on xen-unstable. Tried the same with xen-3.3.0 which is >>>> supposed to have MSI passthrough, but the same guest shows MSI as disabled. >>>> Any else seen this bug, or know of a workaround ? >>>> >>>> Trace is as follows : >>>> >>>> ------------[ cut here ]------------ >>>> kernel BUG at >>>> >>>> >>>> >>>> >>> /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:809> >>> ! >>> >>> >>> >>>> invalid opcode: 0000 [#1] >>>> SMP >>>> Modules linked in: nfemsg nfdvnet ipv6 binfmt_misc dm_mod nfe usbcore >>>> ext3 jbd processor fuse >>>> CPU: 0 >>>> EIP: 0061:[<c02487f5>] Tainted: GF VLI >>>> EFLAGS: 00210097 (2.6.18.8-xen #2) >>>> EIP is at evtchn_get_xen_pirq+0x35/0x40 >>>> eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >>>> esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >>>> ds: 007b es: 007b ss: 0069 >>>> Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 task.ti=ed384000) >>>> Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 >>>> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 >>>> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 >>>> Call Trace: >>>> [<c0248aef>] startup_pirq+0x3f/0x250 >>>> [<c0150b50>] setup_irq+0x160/0x1b0 >>>> [<ee0dd270>] nfe_interrupt_handler+0x0/0x30 [nfemsg] >>>> [<c0150c43>] request_irq+0xa3/0xc0 >>>> [<ee02c8ad>] nfemsg_module_init+0x8ad/0x133e [nfemsg] >>>> [<c030531b>] cond_resched+0x2b/0x40 >>>> [<c0305369>] wait_for_completion+0x19/0xf0 >>>> [<c0142818>] sys_init_module+0x148/0x1b50 >>>> [<c010595f>] syscall_call+0x7/0xb >>>> Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 44 c0 89 >>>> d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 <0f> 0b >>>> 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >>>> EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP 0069:ed385dac >>>> ### card [0] start: host progs ### >>>> -bash-3.2# >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: ------------[ cut here ]------------ >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: kernel BUG at >>>> >>>> >>>> >>>> >>> /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c:809> >>> ! >>> >>> >>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: invalid opcode: 0000 [#1] >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: SMP >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: CPU: 0 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: EIP is at evtchn_get_xen_pirq+0x35/0x40 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: ds: 007b es: 007b ss: 0069 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 >>>> task.ti=ed384000) >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 00000000 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 00000000 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >>>> 00000000 00000000 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: Call Trace: >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 >>>> 44 c0 89 d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 >>>> <0f> 0b 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >>>> >>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>> kernel: EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP >>>> 0069:ed385dac >>>> >>>> >>>> >>> >> -- >> As long as the music''s loud enough, we won''t hear the world falling apart. >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> >> > > > -- > As long as the music''s loud enough, we won''t hear the world falling apart. > >-- As long as the music''s loud enough, we won''t hear the world falling apart. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Possibly, but the code generating that message has changed in current xen-unstable.hg, so you might have more luck with that. -- Keir On 27/9/08 03:41, "Anish Bhatt" <anish@cc.gatech.edu> wrote:> > Works fine now with your patch, but I still see the '' > > physdev.c:90:d0 remap pirq fa vector 49 while not unmap'' message in dom0 xm > dmesg. Is that something to be bothered about ? > -Anish > > Shan, Haitao wrote: >> It is supposed to work on dom0. This is a bug. >> Please refer to Keir''s reply in this mail thread. >> >> Shan Haitao >> >> -----Original Message----- >> From: Anish Bhatt [mailto:anish@cc.gatech.edu] >> Sent: 2008年9月24日 8:08 >> To: Shan, Haitao >> Cc: Jan Beulich; xen-devel@lists.xensource.com; Keir Fraser >> Subject: Re: [Xen-devel] MSI causing softpanics in guest >> >> No, the bug_on is not triggered in dom0. I also saw the following error >> message in xm dmesg, : >> /physdev.c:90:d0 remap pirq fa vector 49 while not unmap/ >> Its part of the code added for MSI, I wonder if its of any significance. >> >> -Anish >> >> Shan, Haitao wrote: >> >>> Just some thoughts on Anish''s problem. It seems he reveals a bug in current >>> code, I think. >>> >>> Currently, evnchn_map_pirq is hooked on msi_map_vector >>> (pci_enable_msi->msi_capability_init->msi_map_vector). Here the pirq type is >>> set to IRQT_PIRQ. Then request_irq should succeed since pirq type is already >>> set. >>> >>> However, this path only works on the assumption that msi_map_vector and >>> request_irq are executed in the same domain, to be more specific, dom0. >>> For PV domU guest, there is no msi_map_vector in its pci_enable_msi ( PV >>> domU actually asks dom0 to do the msi map, instead. This is a decision made >>> by Keir and Yunhong long ago. I think it is base on security >>> considerations). So its irq type is not set correctly (What''s more, it >>> incorrectly sets the dom0''s irq type). Then request_irq can trigger this >>> bug_on(). >>> >>> Anish, if you use the device in dom0 itself, does it also have this bug_on? >>> >>> Keir and Jan, >>> Do you think this explanation is reasonable for issues Anish met? >>> >>> Best Regards >>> Shan Haitao >>> >>> -----Original Message----- >>> From: Jan Beulich [mailto:jbeulich@novell.com] >>> Sent: 2008年9月23日 15:24 >>> To: Anish Bhatt >>> Cc: Keir Fraser; Shan, Haitao; xen-devel@lists.xensource.com >>> Subject: Re: [Xen-devel] MSI causing softpanics in guest >>> >>> So the question is what irq you pass in to request_irq() and where you got >>> it from. Jan >>> >>> >>> >>>>>> Anish Bhatt <anish@gatech.edu> 23.09.08 07:04 >>> >>>>>> >>>>>> >>> I put a printk before BUG_ON, Its expecting IRQT_PIRQ, but getting >>> IRQT_UNBOUND instead. >>> -Anish >>> >>> Jan Beulich wrote: >>> >>> >>>>>>> Keir Fraser <keir.fraser@eu.citrix.com> 20.09.08 10:15 >>> >>>>>>> >>>>>>> >>>>>>> >>>>> On 3.3.0 you need to specify msi=1 on Xen''s command line to enable MSI. In >>>>> xen-unstable, MSI support is always enabled. >>>>> >>>>> I changed a check in evtchn_get_xen_pirq() into a BUG_ON() as it looks to >>>>> me >>>>> like it should never trigger. Jan Beulich was the original authour of that >>>>> function -- cc''ed in case he can indicate whether I''ve actually broken the >>>>> function. :-) >>>>> >>>>> >>>>> >>>> No, I think that BUG_ON() you added is valid. It definitely is in the >>>> context >>>> of startup_pirq(). I''d be curious to know what the type of that irq really >>>> is, >>>> but that cannot be determined from the backtrace alone. Anish, could you >>>> perhaps add a simple printk() before the BUG_ON()? Unfortunately the >>>> driver in question does not appear to be part of the tree, so we can''t even >>>> check what it does prior to calling request_irq()... >>>> >>>> -- Keir >>>> >>>> On 19/9/08 20:53, "Anish Bhatt" <anish@cc.gatech.edu> wrote: >>>> >>>> >>>> >>>> >>>>> lspci shows MSI enabled for PCI device. PCI passthrough works fine. >>>>> However, as soon as the MSI driver for card is insmodded, kernel panics. >>>>> This is on xen-unstable. Tried the same with xen-3.3.0 which is >>>>> supposed to have MSI passthrough, but the same guest shows MSI as >>>>> disabled. >>>>> Any else seen this bug, or know of a workaround ? >>>>> >>>>> Trace is as follows : >>>>> >>>>> ------------[ cut here ]------------ >>>>> kernel BUG at >>>>> >>>>> >>>>> >>>>> >>>> /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c: >>>> 809> >>>> ! >>>> >>>> >>>> >>>>> invalid opcode: 0000 [#1] >>>>> SMP >>>>> Modules linked in: nfemsg nfdvnet ipv6 binfmt_misc dm_mod nfe usbcore >>>>> ext3 jbd processor fuse >>>>> CPU: 0 >>>>> EIP: 0061:[<c02487f5>] Tainted: GF VLI >>>>> EFLAGS: 00210097 (2.6.18.8-xen #2) >>>>> EIP is at evtchn_get_xen_pirq+0x35/0x40 >>>>> eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >>>>> esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >>>>> ds: 007b es: 007b ss: 0069 >>>>> Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 task.ti=ed384000) >>>>> Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 00000000 >>>>> 00000000 >>>>> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >>>>> 00000000 >>>>> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 >>>>> 00000000 >>>>> Call Trace: >>>>> [<c0248aef>] startup_pirq+0x3f/0x250 >>>>> [<c0150b50>] setup_irq+0x160/0x1b0 >>>>> [<ee0dd270>] nfe_interrupt_handler+0x0/0x30 [nfemsg] >>>>> [<c0150c43>] request_irq+0xa3/0xc0 >>>>> [<ee02c8ad>] nfemsg_module_init+0x8ad/0x133e [nfemsg] >>>>> [<c030531b>] cond_resched+0x2b/0x40 >>>>> [<c0305369>] wait_for_completion+0x19/0xf0 >>>>> [<c0142818>] sys_init_module+0x148/0x1b50 >>>>> [<c010595f>] syscall_call+0x7/0xb >>>>> Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 44 c0 89 >>>>> d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 <0f> 0b >>>>> 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >>>>> EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP 0069:ed385dac >>>>> ### card [0] start: host progs ### >>>>> -bash-3.2# >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: ------------[ cut here ]------------ >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: kernel BUG at >>>>> >>>>> >>>>> >>>>> >>>> /usr/src/xen/xen-unstable.hg/linux-2.6.18-xen.hg/drivers/xen/core/evtchn.c: >>>> 809> >>>> ! >>>> >>>> >>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: invalid opcode: 0000 [#1] >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: SMP >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: CPU: 0 >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: EIP is at evtchn_get_xen_pirq+0x35/0x40 >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: eax: ffffffff ebx: 00000002 ecx: c0372e60 edx: 00000000 >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: esi: c2103560 edi: c03d3080 ebp: 000004f9 esp: ed385dac >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: ds: 007b es: 007b ss: 0069 >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: Process modprobe (pid: 2590, ti=ed384000 task=ed7b1100 >>>>> task.ti=ed384000) >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: Stack: c0248aef 00000000 00000000 00000000 00000000 00000000 >>>>> 00000000 00000000 >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >>>>> 00000000 00000000 >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: 00000000 00000000 00000000 00000000 00000000 00000000 >>>>> 00000000 00000000 >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: Call Trace: >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: Code: 00 00 89 d1 d3 e0 85 05 c4 2c 42 c0 74 1a 8b 14 95 c0 c0 >>>>> 44 c0 89 d0 c1 e8 1c 83 e8 01 75 0c c1 ea 0c 81 e2 ff ff 00 00 89 d0 c3 >>>>> <0f> 0b 29 03 a4 f1 32 c0 eb ea 90 83 ec 08 89 74 24 04 89 c6 a1 >>>>> >>>>> Message from syslogd@drake at Sep 19 15:36:44 ... >>>>> kernel: EIP: [<c02487f5>] evtchn_get_xen_pirq+0x35/0x40 SS:ESP >>>>> 0069:ed385dac >>>>> >>>>> >>>>> >>>> >>> -- >>> As long as the music''s loud enough, we won''t hear the world falling apart. >>> >>> >>> _______________________________________________ >>> Xen-devel mailing list >>> Xen-devel@lists.xensource.com >>> http://lists.xensource.com/xen-devel >>> >>> >> >> >> -- >> As long as the music''s loud enough, we won''t hear the world falling apart. >> >> >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Reasonably Related Threads
- [Xen-devel] Crash during boot in Debian lenny default dom0 kernel (2.6.26-2-xen-686) / bugfix patch
- RELENG_7 jerky mouse and skipping sound
- Performance on GlusterFS
- Successful PCIe Graphics VT-d Passthrough to Win32 DomU, Q35 chipset
- Successful PCIe Graphics VT-d Passthrough to Win32 DomU, Q35 chipset