Simon Horman
2009-Sep-23 05:32 UTC
[Xen-devel] pvops: boot locks up while initialising PCI
Hi,
while trying to boot xen/master at the following changest the boot
locks up somewhere around the initialisation of PCI. Boot log and
config attached. Any insights appreciated.
commit 398b4b4cabdb27c06a7b8214c2044347ffc08c49
Merge: 3973dcf... 6e8f158...
Author: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Date: Tue Sep 22 15:21:28 2009 -0700
Merge branch ''xen/dom0/drm'' into xen/master
* xen/dom0/drm:
swiotlb: linux/swiotlb.h needs linux/dma-mapping.h
My grub snippet is:
title Xen
root (hd0,0)
kernel /xen com1=115200,8n1,0x3f8,0 console=com1 loglvl=all
guest_loglvl=all dom0_mem=1G
module /vmlinuz-2.6.31-pvops root=/dev/mapper/sam-root_x86_64
console=ttyS0 loglevel=8 ro earlyprintk=xen
module /initrd.img-2.6.31-pvops
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Pasi Kärkkäinen
2009-Sep-23 07:00 UTC
Re: [Xen-devel] pvops: boot locks up while initialising PCI
On Wed, Sep 23, 2009 at 03:32:07PM +1000, Simon Horman wrote:> > My grub snippet is: > > title Xen > root (hd0,0) > kernel /xen com1=115200,8n1,0x3f8,0 console=com1 loglvl=all guest_loglvl=all dom0_mem=1G > module /vmlinuz-2.6.31-pvops root=/dev/mapper/sam-root_x86_64 console=ttyS0 loglevel=8 ro earlyprintk=xen > module /initrd.img-2.6.31-pvops >Dunno if it makes any difference to your problem, but I''m using "console=hvc0 earlyprintk=xen" and I get full serial console output through Xen. So it''s enough to configure serial console only to Xen (com1=115200,8n1,0x3f8,0 console=com1). -- Pasi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2009-Sep-23 19:17 UTC
[Xen-devel] Re: pvops: boot locks up while initialising PCI
On 09/22/09 22:32, Simon Horman wrote:> Hi, > > while trying to boot xen/master at the following changest the boot > locks up somewhere around the initialisation of PCI. Boot log and > config attached. Any insights appreciated. > > commit 398b4b4cabdb27c06a7b8214c2044347ffc08c49 > Merge: 3973dcf... 6e8f158... > Author: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> > Date: Tue Sep 22 15:21:28 2009 -0700 > > Merge branch ''xen/dom0/drm'' into xen/master > > * xen/dom0/drm: > swiotlb: linux/swiotlb.h needs linux/dma-mapping.h > > > My grub snippet is: > > title Xen > root (hd0,0) > kernel /xen com1=115200,8n1,0x3f8,0 console=com1 loglvl=all guest_loglvl=all dom0_mem=1G > module /vmlinuz-2.6.31-pvops root=/dev/mapper/sam-root_x86_64 console=ttyS0 loglevel=8 ro earlyprintk=xen > module /initrd.img-2.6.31-pvops > >You should really be using hvc0 for the console. I think ttyS0 "passthrough"/emulation should work, but using the Xen console is more reliable. And you''re just losing all console output on the switchover, which suggests that serial isn''t actually working. Failing that, can you get more info from the VGA console? Aside from that, I''m not sure what''s going on. I don''t normally see messages from Xen about PCI devices. Are you using MSI? Does booting with pci=nomsi work? Thanks, J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Simon Horman
2009-Sep-23 23:05 UTC
Re: [Xen-devel] pvops: boot locks up while initialising PCI
On Wed, Sep 23, 2009 at 12:17:23PM -0700, Jeremy Fitzhardinge wrote:> On 09/22/09 22:32, Simon Horman wrote: > > Hi, > > > > while trying to boot xen/master at the following changest the boot > > locks up somewhere around the initialisation of PCI. Boot log and > > config attached. Any insights appreciated. > > > > commit 398b4b4cabdb27c06a7b8214c2044347ffc08c49 > > Merge: 3973dcf... 6e8f158... > > Author: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> > > Date: Tue Sep 22 15:21:28 2009 -0700 > > > > Merge branch ''xen/dom0/drm'' into xen/master > > > > * xen/dom0/drm: > > swiotlb: linux/swiotlb.h needs linux/dma-mapping.h > > > > > > My grub snippet is: > > > > title Xen > > root (hd0,0) > > kernel /xen com1=115200,8n1,0x3f8,0 console=com1 loglvl=all guest_loglvl=all dom0_mem=1G > > module /vmlinuz-2.6.31-pvops root=/dev/mapper/sam-root_x86_64 console=ttyS0 loglevel=8 ro earlyprintk=xen > > module /initrd.img-2.6.31-pvops > > > > > > You should really be using hvc0 for the console. I think ttyS0 > "passthrough"/emulation should work, but using the Xen console is more > reliable. And you''re just losing all console output on the switchover, > which suggests that serial isn''t actually working. Failing that, can > you get more info from the VGA console?Hi Jeremy, Hi Pasi, Thanks for the advice with regards to hvc0, that is working well. I am now observing a problem while initialising the iommu. I have worked around this by setting iommu=off. [ 0.368001] DMAR:No ATSR found [ 0.368104] IOMMU 0xfed92000: using Register based invalidation [ 0.372001] IOMMU 0xfed91000: using Register based invalidation [ 0.372001] IOMMU 0xfed90000: using Register based invalidation [ 0.372001] IOMMU 0xfed93000: using Register based invalidation [ 0.372001] IOMMU: Setting RMRR: [ 0.372001] IOMMU: Setting identity map for device 0000:00:1a.1 [0xdefd6000 - 0xdefd7000] [ 0.372001] IOMMU: Setting identity map for device 0000:00:1a.0 [0xdefd5000 - 0xdefd6000] [ 0.372001] IOMMU: Setting identity map for device 0000:00:1d.2 [0xdefd4000 - 0xdefd5000] [ 0.372001] IOMMU: Setting identity map for device 0000:00:1d.1 [0xdefd3000 - 0xdefd4000] [ 0.372001] IOMMU: Setting identity map for device 0000:00:1d.0 [0xdefd2000 - 0xdefd3000] [ 0.372001] IOMMU: Setting identity map for device 0000:00:1a.7 [0xdefd1000 - 0xdefd2000] [ 0.372001] IOMMU: Setting identity map for device 0000:00:1d.7 [0xdefd0000 - 0xdefd1000] [ 0.372001] IOMMU: Setting identity map for device 0000:00:02.0 [0xdf600000 - 0xe0000000] [ 0.372001] IOMMU: Prepare 0-16MiB unity mapping for LPC [ 0.372001] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0x1000000] [ 0.372001] BUG: unable to handle kernel NULL pointer dereference at 000000000000008c [ 0.372001] IP: [<ffffffff810273f4>] create_irq_nr+0x56/0xc6 [ 0.372001] PGD 0 [ 0.372001] Oops: 0000 [#1] SMP [ 0.372001] last sysfs file: [ 0.372001] CPU 0 [ 0.372001] Modules linked in: [ 0.372001] Pid: 1, comm: swapper Not tainted 2.6.31-pvops #13 HP Compaq dc7800p Convertible Minitower [ 0.372001] RIP: e030:[<ffffffff810273f4>] [<ffffffff810273f4>] create_irq_nr+0x56/0xc6 [ 0.372001] RSP: e02b:ffff88003fea3e20 EFLAGS: 00010086 [ 0.372001] RAX: ffff88003fc5ca00 RBX: 0000000000000018 RCX: 0000000000000000 [ 0.372001] RDX: 0000000000000218 RSI: ffff88003f4ee7d0 RDI: ffff88003f4d3000 [ 0.372001] RBP: ffff88003fc5ca00 R08: ffff88003feddd00 R09: 0000000000000000 [ 0.372001] R10: 0000000000000000 R11: 0000000000000002 R12: 0000000000000000 [ 0.372001] R13: 0000000000000000 R14: 0000000000000218 R15: 0000000000000000 [ 0.372001] FS: 0000000000000000(0000) GS:ffffc90000000000(0000) knlGS:0000000000000000 [ 0.372001] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b [ 0.372001] CR2: 000000000000008c CR3: 0000000001001000 CR4: 0000000000002660 [ 0.372001] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 0.372001] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 0.372001] Process swapper (pid: 1, threadinfo ffff88003fea2000, task ffff88003fea1480) [ 0.372001] Stack: [ 0.372001] ffff88003f4ee840 0000000000000000 ffff88003f4ee840 ffffffff813dac10 [ 0.372001] <0> 0000000000000000 ffffffff810276bc ffff88003f4ee840 ffffffff811762d2 [ 0.372001] <0> ffff88003f4ee840 0000000000000001 ffff88003feddd80 ffffffff81444952 [ 0.372001] Call Trace: [ 0.372001] [<ffffffff810276bc>] ? create_irq+0x28/0x35 [ 0.372001] [<ffffffff811762d2>] ? dmar_set_interrupt+0x18/0x9f [ 0.372001] [<ffffffff81444952>] ? init_dmars+0x57a/0x643 [ 0.372001] [<ffffffff81170017>] ? pci_get_subsys+0x60/0x73 [ 0.372001] [<ffffffff81444c91>] ? intel_iommu_init+0x276/0x369 [ 0.372001] [<ffffffff81427492>] ? pci_iommu_init+0x0/0x26 [ 0.372001] [<ffffffff814274a5>] ? pci_iommu_init+0x13/0x26 [ 0.372001] [<ffffffff81451626>] ? sysctl_init+0x0/0x3f [ 0.372001] [<ffffffff8100a065>] ? do_one_initcall+0x64/0x17d [ 0.372001] [<ffffffff81421127>] ? kernel_init+0x168/0x1be [ 0.372001] [<ffffffff81013b0a>] ? child_rip+0xa/0x20 [ 0.372001] [<ffffffff81012ce7>] ? int_ret_from_sys_call+0x7/0x1b [ 0.372001] [<ffffffff8101349d>] ? retint_restore_args+0x5/0x6 [ 0.372001] [<ffffffff811ad3d8>] ? acpi_tb_verify_table+0x21/0x4f [ 0.372001] [<ffffffff811ad3d8>] ? acpi_tb_verify_table+0x21/0x4f [ 0.372001] [<ffffffff81013b00>] ? child_rip+0x0/0x20 [ 0.372001] Code: c6 eb 60 44 89 ee 89 df e8 c3 5d 25 00 48 85 c0 48 89 c5 75 12 89 de 48 c7 c7 88 1a 32 81 31 c0 e8 cc 52 26 00 eb 3a 4c 8b 60 40 <41> 80 bc 24 8c 00 00 00 00 75 2b 44 89 ee 48 89 c7 e8 3a 44 06 [ 0.372001] RIP [<ffffffff810273f4>] create_irq_nr+0x56/0xc6 [ 0.372001] RSP <ffff88003fea3e20> [ 0.372001] CR2: 000000000000008c [ 0.372001] ---[ end trace 4eaa2a86a8e2da22 ]--- [ 0.372006] Kernel panic - not syncing: Attempted to kill init! [ 0.376001] Pid: 1, comm: swapper Tainted: G D 2.6.31-pvops #13 [ 0.376001] Call Trace: [ 0.376001] [<ffffffff8128c606>] ? panic+0x86/0x13a [ 0.376001] [<ffffffff8100922a>] ? _stext+0x22a/0x1001 [ 0.376001] [<ffffffff8100e839>] ? xen_irq_enable_direct_end+0x0/0x7 [ 0.376001] [<ffffffff8100e219>] ? xen_force_evtchn_callback+0x9/0xa [ 0.376001] [<ffffffff8100e892>] ? check_events+0x12/0x20 [ 0.376001] [<ffffffff8128e325>] ? _write_lock_irq+0x7/0x16 [ 0.376001] [<ffffffff81052902>] ? exit_ptrace+0xa7/0x126 [ 0.376001] [<ffffffff8104cdb6>] ? do_exit+0x72/0x698 [ 0.376001] [<ffffffff8100e892>] ? check_events+0x12/0x20 [ 0.376001] [<ffffffff81016a16>] ? oops_end+0xaf/0xb4 [ 0.376001] [<ffffffff81030e57>] ? no_context+0x1ec/0x1fb [ 0.376001] [<ffffffff81030ff3>] ? __bad_area_nosemaphore+0x18d/0x1b1 [ 0.376001] [<ffffffff810a7704>] ? __alloc_pages_nodemask+0xf8/0x53c [ 0.376001] [<ffffffff810311a7>] ? do_page_fault+0x29/0x20f [ 0.376001] [<ffffffff8128e725>] ? page_fault+0x25/0x30 [ 0.376001] [<ffffffff810273f4>] ? create_irq_nr+0x56/0xc6 [ 0.376001] [<ffffffff810276bc>] ? create_irq+0x28/0x35 [ 0.376001] [<ffffffff811762d2>] ? dmar_set_interrupt+0x18/0x9f [ 0.376001] [<ffffffff81444952>] ? init_dmars+0x57a/0x643 [ 0.376001] [<ffffffff81170017>] ? pci_get_subsys+0x60/0x73 [ 0.376001] [<ffffffff81444c91>] ? intel_iommu_init+0x276/0x369 [ 0.376001] [<ffffffff81427492>] ? pci_iommu_init+0x0/0x26 [ 0.376001] [<ffffffff814274a5>] ? pci_iommu_init+0x13/0x26 [ 0.376001] [<ffffffff81451626>] ? sysctl_init+0x0/0x3f [ 0.376001] [<ffffffff8100a065>] ? do_one_initcall+0x64/0x17d [ 0.376001] [<ffffffff81421127>] ? kernel_init+0x168/0x1be [ 0.376001] [<ffffffff81013b0a>] ? child_rip+0xa/0x20 [ 0.376001] [<ffffffff81012ce7>] ? int_ret_from_sys_call+0x7/0x1b [ 0.376001] [<ffffffff8101349d>] ? retint_restore_args+0x5/0x6 [ 0.376001] [<ffffffff811ad3d8>] ? acpi_tb_verify_table+0x21/0x4f [ 0.376001] [<ffffffff811ad3d8>] ? acpi_tb_verify_table+0x21/0x4f [ 0.376001] [<ffffffff81013b00>] ? child_rip+0x0/0x20 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2009-Sep-23 23:29 UTC
Re: [Xen-devel] pvops: boot locks up while initialising PCI
On 09/23/09 16:05, Simon Horman wrote:> On Wed, Sep 23, 2009 at 12:17:23PM -0700, Jeremy Fitzhardinge wrote: > >> On 09/22/09 22:32, Simon Horman wrote: >> >>> Hi, >>> >>> while trying to boot xen/master at the following changest the boot >>> locks up somewhere around the initialisation of PCI. Boot log and >>> config attached. Any insights appreciated. >>> >>> commit 398b4b4cabdb27c06a7b8214c2044347ffc08c49 >>> Merge: 3973dcf... 6e8f158... >>> Author: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> >>> Date: Tue Sep 22 15:21:28 2009 -0700 >>> >>> Merge branch ''xen/dom0/drm'' into xen/master >>> >>> * xen/dom0/drm: >>> swiotlb: linux/swiotlb.h needs linux/dma-mapping.h >>> >>> >>> My grub snippet is: >>> >>> title Xen >>> root (hd0,0) >>> kernel /xen com1=115200,8n1,0x3f8,0 console=com1 loglvl=all guest_loglvl=all dom0_mem=1G >>> module /vmlinuz-2.6.31-pvops root=/dev/mapper/sam-root_x86_64 console=ttyS0 loglevel=8 ro earlyprintk=xen >>> module /initrd.img-2.6.31-pvops >>> >>> >>> >> You should really be using hvc0 for the console. I think ttyS0 >> "passthrough"/emulation should work, but using the Xen console is more >> reliable. And you''re just losing all console output on the switchover, >> which suggests that serial isn''t actually working. Failing that, can >> you get more info from the VGA console? >> > Hi Jeremy, Hi Pasi, > > Thanks for the advice with regards to hvc0, that is working well. > I am now observing a problem while initialising the iommu. > I have worked around this by setting iommu=off. >Ah, OK, someone else noticed that too. I guess we should inhibit iommu init somehow. Keir, is Xen supposed to disable the DMAR for dom0 by stomping on acpi or something? Or was that about something else? J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2009-Sep-24 07:26 UTC
Re: [Xen-devel] pvops: boot locks up while initialising PCI
On 24/09/2009 00:29, "Jeremy Fitzhardinge" <jeremy@goop.org> wrote:>> Thanks for the advice with regards to hvc0, that is working well. >> I am now observing a problem while initialising the iommu. >> I have worked around this by setting iommu=off. >> > > Ah, OK, someone else noticed that too. I guess we should inhibit iommu > init somehow. > > Keir, is Xen supposed to disable the DMAR for dom0 by stomping on acpi > or something? Or was that about something else?Xen does this. In 3.4 and earlier it disables it only if Xen has iommu enabled itself. In xen-unstable it nobbles DMAR tables in all cases -- presumably this is Intel''s fix for the very issue you are talking about. Probably Intel''s fix needs backport to 3.4? -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Han, Weidong
2009-Sep-24 07:28 UTC
RE: [Xen-devel] pvops: boot locks up while initialising PCI
Jeremy Fitzhardinge wrote:> On 09/23/09 16:05, Simon Horman wrote: >> On Wed, Sep 23, 2009 at 12:17:23PM -0700, Jeremy Fitzhardinge wrote: >> >>> On 09/22/09 22:32, Simon Horman wrote: >>> >>>> Hi, >>>> >>>> while trying to boot xen/master at the following changest the boot >>>> locks up somewhere around the initialisation of PCI. Boot log and >>>> config attached. Any insights appreciated. >>>> >>>> commit 398b4b4cabdb27c06a7b8214c2044347ffc08c49 >>>> Merge: 3973dcf... 6e8f158... >>>> Author: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> >>>> Date: Tue Sep 22 15:21:28 2009 -0700 >>>> >>>> Merge branch ''xen/dom0/drm'' into xen/master >>>> >>>> * xen/dom0/drm: >>>> swiotlb: linux/swiotlb.h needs linux/dma-mapping.h >>>> >>>> >>>> My grub snippet is: >>>> >>>> title Xen >>>> root (hd0,0) >>>> kernel /xen com1=115200,8n1,0x3f8,0 console=com1 >>>> loglvl=all guest_loglvl=all dom0_mem=1G >>>> module /vmlinuz-2.6.31-pvops >>>> root=/dev/mapper/sam-root_x86_64 console=ttyS0 loglevel=8 ro >>>> earlyprintk=xen module /initrd.img-2.6.31-pvops >>>> >>>> >>>> >>> You should really be using hvc0 for the console. I think ttyS0 >>> "passthrough"/emulation should work, but using the Xen console is >>> more reliable. And you''re just losing all console output on the >>> switchover, which suggests that serial isn''t actually working. >>> Failing that, can you get more info from the VGA console? >>> >> Hi Jeremy, Hi Pasi, >> >> Thanks for the advice with regards to hvc0, that is working well. >> I am now observing a problem while initialising the iommu. >> I have worked around this by setting iommu=off. >> > > Ah, OK, someone else noticed that too. I guess we should inhibit > iommu init somehow. > > Keir, is Xen supposed to disable the DMAR for dom0 by stomping on acpi > or something? Or was that about something else? >Now Xen will zap ACPI DMAR signature to prevent dom0 using VT-d. pv-ops dom0 shouldn''t use VT-d. Simon, I noticed you use xen c/s 20010, pls try xen c/s 20181. Regards, Weidong Regards, Weidong> J > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Simon Horman
2009-Sep-25 00:05 UTC
Re: [Xen-devel] pvops: boot locks up while initialising PCI
On Thu, Sep 24, 2009 at 03:28:28PM +0800, Han, Weidong wrote:> > Now Xen will zap ACPI DMAR signature to prevent dom0 using VT-d. pv-ops dom0 shouldn''t use VT-d. > > Simon, I noticed you use xen c/s 20010, pls try xen c/s 20181.Thanks, I have confirmed that updating xen resolves the problem that I was seeing. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel