Current Xend allowing multiple call destroy() for same domain, this lead multiple hard resets(FLR) for pci pass-through, and some controller might failed. In our test, we pass through 2 LSI HAB controllers to the PVHVM guest, after guest brought up, call xm-destroy twice, the adapters''s BIOS will hung, and we had to reboot the server to recovery it. Signed-off-by: Joe Jin <joe.jin@oracle.com> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Cc: Ian Campbell <ian.campbell@citrix.com> --- tools/python/xen/xend/XendDomainInfo.py | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/tools/python/xen/xend/XendDomainInfo.py b/tools/python/xen/xend/XendDomainInfo.py index e9d3e7e..658c3b6 100644 --- a/tools/python/xen/xend/XendDomainInfo.py +++ b/tools/python/xen/xend/XendDomainInfo.py @@ -367,6 +367,8 @@ class XendDomainInfo: @type refresh_shutdown_lock: threading.Condition @ivar _deviceControllers: device controller cache for this domain @type _deviceControllers: dict ''string'' to DevControllers + @ivar destroying: Is this domain destroying + @type destroying: bool """ def __init__(self, info, domid = None, dompath = None, augment = False, @@ -455,6 +457,8 @@ class XendDomainInfo: self._checkName(self.info[''name_label'']) self.metrics = XendVMMetrics(uuid.createString(), self) + + self.destroying = False # @@ -3073,6 +3077,12 @@ class XendDomainInfo: if self.domid is None: return + + if self.destroying == False: + self.destroying = True + else: + raise VmError("Domain (domid=%s) is destroying, please wait!", str(self.domid)) + from xen.xend import XendDomain log.debug("XendDomainInfo.destroy: domid=%s", str(self.domid)) @@ -3088,6 +3098,7 @@ class XendDomainInfo: self.info[state] = 0 self._stateSet(DOM_STATE_HALTED) except: + self.destroying = False log.exception("XendDomainInfo.destroy: domain destruction failed.") XendDomain.instance().remove_domain(self) -- 1.8.1
On 04/02/13 02:54, Joe Jin wrote:> Current Xend allowing multiple call destroy() for same domain, this lead > multiple hard resets(FLR) for pci pass-through, and some controller might > failed. > > In our test, we pass through 2 LSI HAB controllers to the PVHVM guest, after > guest brought up, call xm-destroy twice, the adapters''s BIOS will hung, and > we had to reboot the server to recovery it.Does the same problem happen with libxl/xl?
On 02/04/13 17:52, Roger Pau Monné wrote:> On 04/02/13 02:54, Joe Jin wrote: >> Current Xend allowing multiple call destroy() for same domain, this lead >> multiple hard resets(FLR) for pci pass-through, and some controller might >> failed. >> >> In our test, we pass through 2 LSI HAB controllers to the PVHVM guest, after >> guest brought up, call xm-destroy twice, the adapters''s BIOS will hung, and >> we had to reboot the server to recovery it. > > Does the same problem happen with libxl/xl?execute xl-destroy twice crashed my server! Thanks, Joe
On Tue, 2013-02-05 at 06:47 +0000, Joe Jin wrote:> execute xl-destroy twice crashed my server!Can you give more details please. Ian.
On 02/05/13 17:42, Ian Campbell wrote:> On Tue, 2013-02-05 at 06:47 +0000, Joe Jin wrote: >> > execute xl-destroy twice crashed my server! > Can you give more details please. >Hi Ian, Sorry the server not available for test now. Would you please help to review my patch for xm-destroy? Thanks, Joe
I got serial console output now and mostly like panic happened by "xl create": (XEN) HVM1: Detected Xen v4.1.2-OVM (XEN) HVM1: CPU speed is 2926 MHz (XEN) HVM1: Xenbus rings @0xfeffc000, event channel 13 (XEN) irq.c:264: Dom1 PCI link 0 changed 0 -> 5 (XEN) HVM1: PCI-ISA link 0 routed to IRQ5 (XEN) irq.c:264: Dom1 PCI link 1 changed 0 -> 10 (XEN) HVM1: PCI-ISA link 1 routed to IRQ10 (XEN) irq.c:264: Dom1 PCI link 2 changed 0 -> 11 (XEN) HVM1: PCI-ISA link 2 routed to IRQ11 (XEN) irq.c:264: Dom1 PCI link 3 changed 0 -> 5 (XEN) HVM1: PCI-ISA link 3 routed to IRQ5 (XEN) HVM1: pci dev 01:3 INTA->IRQ10 (XEN) HVM1: pci dev 03:0 INTA->IRQ5 (XEN) HVM1: pci dev 04:0 INTA->IRQ5 (XEN) HVM1: pci dev 05:0 INTA->IRQ10 (XEN) HVM1: pci dev 02:0 bar 10 size 02000000: f0000008 (XEN) HVM1: pci dev 03:0 bar 14 size 01000000: f2000008 (XEN) HVM1: pci dev 04:0 bar 30 size 00080000: f3000000 (XEN) HVM1: pci dev 05:0 bar 30 size 00080000: f3080000 (XEN) domctl.c:985:d0 memory_map:add: gfn=f3100 mfn=df040 nr_mfns=40 (XEN) HVM1: pci dev 04:0 bar 1c size 00040000: f3100004 (XEN) domctl.c:985:d0 memory_map:add: gfn=f3140 mfn=df640 nr_mfns=40 (XEN) HVM1: pci dev 05:0 bar 1c size 00040000: f3140004 (XEN) domctl.c:985:d0 memory_map:add: gfn=f3180 mfn=df03c nr_mfns=4 (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3182 mfn=df03e nr_mfns=1 (XEN) HVM1: pci dev 04:0 bar 14 size 00004000: f3180004 (XEN) domctl.c:985:d0 memory_map:add: gfn=f3184 mfn=df63c nr_mfns=4 (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3186 mfn=df63e nr_mfns=1 (XEN) HVM1: pci dev 05:0 bar 14 size 00004000: f3184004 (XEN) HVM1: pci dev 02:0 bar 14 size 00001000: f3188000 (XEN) HVM1: pci dev 03:0 bar 10 size 00000100: 0000c001 (XEN) HVM1: pci dev 04:0 bar 10 size 00000100: 0000c101 (XEN) domctl.c:1041:d0 ioport_map:add f_gport=c100 f_mport=9000 np=100 (XEN) HVM1: pci dev 05:0 bar 10 size 00000100: 0000c201 (XEN) domctl.c:1041:d0 ioport_map:add f_gport=c200 f_mport=d000 np=100 (XEN) HVM1: pci dev 01:1 bar 20 size 00000010: 0000c301 (XEN) HVM1: Multiprocessor initialisation: (XEN) HVM1: - CPU0 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (XEN) HVM1: - CPU1 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (XEN) HVM1: - CPU2 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (XEN) HVM1: - CPU3 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (XEN) HVM1: - CPU4 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (XEN) HVM1: - CPU5 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (XEN) HVM1: - CPU6 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (XEN) HVM1: - CPU7 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (XEN) HVM1: - CPU8 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (XEN) HVM1: - CPU9 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (XEN) HVM1: - CPU10 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (XEN) HVM1: - CPU11 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (XEN) HVM1: Writing SMBIOS tables ... (XEN) HVM1: Loading ROMBIOS ... (XEN) HVM1: 10364 bytes of ROMBIOS high-memory extensions: (XEN) HVM1: Relocating to 0xfc000000-0xfc00287c ... done (XEN) HVM1: Creating MP tables ... (XEN) HVM1: Loading Cirrus VGABIOS ... (XEN) domctl.c:985:d0 memory_map:add: gfn=f3000 mfn=df080 nr_mfns=80 (XEN) HVM1: Loading PCI Option ROM ... (XEN) HVM1: - Manufacturer: LSI Corporation (XEN) HVM1: - Product name: LSI MPI Boot Support (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3000 mfn=df080 nr_mfns=80 (XEN) domctl.c:985:d0 memory_map:add: gfn=f3080 mfn=df680 nr_mfns=80 (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3080 mfn=df680 nr_mfns=80 (XEN) HVM1: Loading ACPI ... (XEN) HVM1: - Lo data: 000ea020-000ea04f (XEN) HVM1: - Hi data: fc002c00-fc00eb4f (XEN) HVM1: vm86 TSS at fc00ec00 (XEN) HVM1: BIOS map: (XEN) HVM1: c0000-c8fff: VGA BIOS (XEN) HVM1: c9000-d4fff: PCI Option ROMs (XEN) HVM1: eb000-eb302: SMBIOS tables (XEN) HVM1: f0000-fffff: Main BIOS (XEN) HVM1: E820 table: (XEN) HVM1: [00]: 00000000:00000000 - 00000000:0009e000: RAM (XEN) HVM1: [01]: 00000000:0009e000 - 00000000:0009fc00: RESERVED (XEN) HVM1: [02]: 00000000:0009fc00 - 00000000:000a0000: RESERVED (XEN) HVM1: HOLE: 00000000:000a0000 - 00000000:000e0000 (XEN) HVM1: [03]: 00000000:000e0000 - 00000000:00100000: RESERVED (XEN) HVM1: [04]: 00000000:00100000 - 00000000:f0000000: RAM (XEN) HVM1: HOLE: 00000000:f0000000 - 00000000:fc000000 (XEN) HVM1: [05]: 00000000:fc000000 - 00000001:00000000: RESERVED (XEN) HVM1: [06]: 00000001:00000000 - 00000004:0f800000: RAM (XEN) HVM1: Invoking ROMBIOS ... (XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $ (XEN) stdvga.c:147:d1 entering stdvga and caching modes (XEN) HVM1: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert Exp $ (XEN) HVM1: Bochs BIOS - build: 06/23/99 (XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $ (XEN) HVM1: Options: apmbios pcibios eltorito PMM (XEN) HVM1: (XEN) HVM1: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 (XEN) HVM1: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (60000 MBytes) (XEN) HVM1: ata0-1: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 (XEN) HVM1: ata0 slave: QEMU HARDDISK ATA-7 Hard-Disk ( 97 GBytes) (XEN) HVM1: ata1-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 (XEN) HVM1: ata1 master: QEMU HARDDISK ATA-7 Hard-Disk (20000 MBytes) (XEN) HVM1: IDE time out (XEN) HVM1: (XEN) HVM1: PCI device 1000:0070 not found at index 0 (XEN) HVM1: PCI device 1000:0072 not found at index 2 (XEN) HVM1: PCI device 1000:0074 not found at index 0 (XEN) HVM1: PCI device 1000:0076 not found at index 0 (XEN) HVM1: PCI device 1000:0077 not found at index 0 (XEN) HVM1: PCI device 1000:0064 not found at index 0 (XEN) HVM1: PCI device 1000:0065 not found at index 0 (XEN) HVM1: PCI device 1000:0080 not found at index 0 (XEN) HVM1: PCI device 1000:0081 not found at index 0 (XEN) HVM1: PCI device 1000:0082 not found at index 0 (XEN) HVM1: PCI device 1000:0083 not found at index 0 (XEN) HVM1: PCI device 1000:0084 not found at index 0 (XEN) HVM1: PCI device 1000:0085 not found at index 0 (XEN) HVM1: PCI device 1000:0086 not found at index 0 (XEN) HVM1: PCI device 1000:0087 not found at index 0 (XEN) HVM1: PCI device 1000:0070 not found at index 0 (XEN) HVM1: PCI device 1000:0072 not found at index 2 (XEN) HVM1: PCI device 1000:0074 not found at index 0 (XEN) HVM1: PCI device 1000:0076 not found at index 0 (XEN) HVM1: PCI device 1000:0077 not found at index 0 (XEN) HVM1: PCI device 1000:0064 not found at index 0 (XEN) HVM1: PCI device 1000:0065 not found at index 0 (XEN) HVM1: PCI device 1000:0080 not found at index 0 (XEN) HVM1: PCI device 1000:0081 not found at index 0 (XEN) HVM1: PCI device 1000:0082 not found at index 0 (XEN) HVM1: PCI device 1000:0083 not found at index 0 (XEN) HVM1: PCI device 1000:0084 not found at index 0 (XEN) HVM1: PCI device 1000:0085 not found at index 0 (XEN) HVM1: PCI device 1000:0086 not found at index 0 (XEN) HVM1: PCI device 1000:0087 not found at index 0 (XEN) HVM1: (XEN) HVM1: (XEN) HVM1: Press F12 for boot menu. (XEN) HVM1: (XEN) HVM1: Booting from Hard Disk... (XEN) HVM1: Booting from 0000:7c00 (XEN) HVM1: *** int 15h function AX=00c0, BX=0000 not yet supported! (XEN) HVM1: *** int 15h function AX=ec00, BX=0002 not yet supported! (XEN) HVM1: KBD: unsupported int 16h function 03 (XEN) HVM1: *** int 15h function AX=e980, BX=0000 not yet supported! (XEN) irq.c:330: Dom1 callback via changed to Direct Vector 0xe9 (XEN) domctl.c:1065:d0 ioport_map:remove f_gport=c100 f_mport=9000 np=100 (XEN) domctl.c:1041:d0 ioport_map:add f_gport=c100 f_mport=9000 np=100 (XEN) domctl.c:1065:d0 ioport_map:remove f_gport=c200 f_mport=d000 np=100 (XEN) domctl.c:1041:d0 ioport_map:add f_gport=c200 f_mport=d000 np=100 (XEN) irq.c:264: Dom1 PCI link 0 changed 5 -> 0 (XEN) irq.c:264: Dom1 PCI link 1 changed 10 -> 0 (XEN) irq.c:264: Dom1 PCI link 2 changed 11 -> 0 (XEN) irq.c:264: Dom1 PCI link 3 changed 5 -> 0 ------------[ cut here ]------------ WARNING: at fs/proc/generic.c:850 remove_proc_entry+0x22d/0x240() Hardware name: SUN FIRE X4370 M2 SERVER remove_proc_entry: removing non-empty directory ''irq/536'', leaking at least ''eth7'' Modules linked in: xt_physdev iptable_filter ip_tables xen_blkback xen_netback xen_pciback xen_gntdev xen_evtchn ipmi_devintf ipmi_si lockd sunrpc bridge stp llc bonding dm_round_robin dm_multipath be2iscsi iscsi_boot_sysfs ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic uio ipv6 cxgb3i libcxgbi cxgb3 libiscsi_tcp libiscsi scsi_transport_iscsi xenfs xen_privcmd dm_mirror video sbs sbshc acpi_memhotplug acpi_ipmi ipmi_msghandler parport_pc lp parport ses enclosure ixgbe igb e1000e mdio snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc i2c_i801 i2c_core i7core_edac iTCO_wdt pcspkr ioatdma iTCO_vendor_support edac_core dca ghes hed dm_region_hash dm_log dm_mod usb_storage s hpchp sg ahci libahci sd_mod crc_t10dif raid1 ext3 jbd mbcache [last unloaded: raid_class] Pid: 19329, comm: qemu-dm Not tainted 2.6.39-200.1.15.el5uek #1 Call Trace: [<ffffffff811d10dd>] ? remove_proc_entry+0x22d/0x240 [<ffffffff8106de10>] warn_slowpath_common+0x90/0xc0 [<ffffffff8106df3e>] warn_slowpath_fmt+0x6e/0x70 [<ffffffff8100a7cf>] ? xen_restore_fl_direct_reloc+0x4/0x4 [<ffffffff8125cf98>] ? sprintf+0x68/0x70 [<ffffffff811cfef1>] ? __xlate_proc_name+0x41/0xc0 [<ffffffff81501b9e>] ? _raw_spin_lock+0xe/0x20 [<ffffffff811d10dd>] remove_proc_entry+0x22d/0x240 [<ffffffff8113232c>] ? zap_pte_range+0x1ec/0x400 [<ffffffff810d9430>] unregister_irq_proc+0xd0/0xf0 [<ffffffff810d571a>] free_desc+0x2a/0x70 [<ffffffff810d579e>] irq_free_descs+0x3e/0x80 [<ffffffff812f7822>] xen_free_irq+0x52/0x70 [<ffffffff812f82cb>] unbind_from_irq+0xeb/0x180 [<ffffffff81009f0d>] ? xen_force_evtchn_callback+0xd/0x10 [<ffffffff812f837f>] evtchn_put+0x1f/0x40 [<ffffffffa055b4e9>] gntdev_put_map+0x39/0x110 [xen_gntdev] [<ffffffff812640e1>] ? list_del+0x11/0x40 [<ffffffffa055c38c>] gntdev_ioctl_unmap_grant_ref+0xac/0xd0 [xen_gntdev] [<ffffffffa055c708>] gntdev_ioctl+0x98/0xa0 [xen_gntdev] [<ffffffff8117eb8d>] vfs_ioctl+0x1d/0x50 [<ffffffff8117f623>] do_vfs_ioctl+0x63/0x1b0 [<ffffffff8113a740>] ? do_munmap+0x240/0x280 [<ffffffff8117f804>] sys_ioctl+0x94/0xa0 [<ffffffff8150a242>] system_call_fastpath+0x16/0x1b ---[ end trace fa61ba514bac93c7 ]--- BUG: unable to handle kernel NULL pointer dereference at 000000000000001c IP: [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40 PGD 6a2cb067 PUD 71d8c067 PMD 0 Oops: 0000 [#1] SMP CPU 0 Modules linked in: xt_physdev iptable_filter ip_tables xen_blkback xen_netback xen_pciback xen_gntdev xen_evtchn ipmi_devintf ipmi_si lockd sunrpc bridge stp llc bonding dm_round_robin dm_multipath be2iscsi iscsi_boot_sysfs ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic uio ipv6 cxgb3i libcxgbi cxgb3 libiscsi_tcp libiscsi scsi_transport_iscsi xenfs xen_privcmd dm_mirror video sbs sbshc acpi_memhotplug acpi_ipmi ipmi_msghandler parport_pc lp parport ses enclosure ixgbe igb e1000e mdio snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc i2c_i801 i2c_core i7core_edac iTCO_wdt pcspkr ioatdma iTCO_vendor_support edac_core dca ghes hed dm_region_hash dm_log dm_mod usb_storage s hpchp sg ahci libahci sd_mod crc_t10dif raid1 ext3 jbd mbcache [last unloaded: raid_class] Pid: 19329, comm: qemu-dm Tainted: G W 2.6.39-200.1.15.el5uek #1 ORACLE CORPORATION SUN FIRE X4370 M2 SERVER /ASSY,SC_BD,T4 RIP: e030:[<ffffffff812f6a76>] [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40 RSP: e02b:ffff88006cfc5dd8 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffff88007e38be48 RDX: 0000000000000000 RSI: 0000000000000218 RDI: ffff88007e38bd70 RBP: ffff88006cfc5dd8 R08: 0000000000000001 R09: 0000000000000000 R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000218 R13: 0000000000000000 R14: 0000000000000017 R15: 0000000000104701 FS: 00007ff36f4046e0(0000) GS:ffff88007f49a000(0000) knlGS:0000000000000000 CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 000000000000001c CR3: 000000005f445000 CR4: 0000000000002660 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process qemu-dm (pid: 19329, threadinfo ffff88006cfc4000, task ffff880067084200) Stack: ffff88006cfc5e38 ffffffff812f81fc 00007ff36e620000 0000003e38e10000 ffff88006cfc5e38 000000000000014f 0000000000000206 ffff88006b2e8138 ffffffff81009f0d ffff88006b951cc0 ffff88006b951cc0 0000000000000000 Call Trace: [<ffffffff812f81fc>] unbind_from_irq+0x1c/0x180 [<ffffffff81009f0d>] ? xen_force_evtchn_callback+0xd/0x10 [<ffffffff812f837f>] evtchn_put+0x1f/0x40 [<ffffffffa055b4e9>] gntdev_put_map+0x39/0x110 [xen_gntdev] [<ffffffff812640e1>] ? list_del+0x11/0x40 [<ffffffffa055c38c>] gntdev_ioctl_unmap_grant_ref+0xac/0xd0 [xen_gntdev] [<ffffffffa055c708>] gntdev_ioctl+0x98/0xa0 [xen_gntdev] [<ffffffff8117eb8d>] vfs_ioctl+0x1d/0x50 [<ffffffff8117f623>] do_vfs_ioctl+0x63/0x1b0 [<ffffffff8113a740>] ? do_munmap+0x240/0x280 [<ffffffff8117f804>] sys_ioctl+0x94/0xa0 [<ffffffff8150a242>] system_call_fastpath+0x16/0x1b Code: 8b 64 24 08 c9 c3 0f 1f 80 00 00 00 00 0f 1f 84 00 00 00 00 00 55 48 89 e5 66 66 66 66 90 39 3d f1 16 4c 00 76 0b e8 ea fa ff ff <0f> b7 40 1c c9 c3 89 f9 31 c0 48 c7 c2 2f 72 70 81 be d1 00 00 RIP [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40 RSP <ffff88006cfc5dd8> CR2: 000000000000001c Thanks, Joe On 02/05/13 17:42, Ian Campbell wrote:> On Tue, 2013-02-05 at 06:47 +0000, Joe Jin wrote: >> execute xl-destroy twice crashed my server! > > Can you give more details please. > > Ian. > > >
Execute "xl-destroy" twice lead the dom0 panic: config file as below: vncunused = 1 kernel = ''/usr/lib/xen/boot/hvmloader'' vnc = 1 name = ''test'' memory = ''16384'' #memory = ''1024'' timer_mode = 0 device_model = ''/usr/lib64/xen/bin/qemu-dm'' builder = ''hvm'' vnclisten = ''0.0.0.0'' cpus = ''0,2,4,6,8,11,1,3,5,7,9,10'' #on_crash = ''coredump-restart'' #on_crash = ''preserve'' on_crash = ''destroy'' on_reboot = ''restart'' vcpus = 12 pci = [''0d:00.0'', ''1f:00.0''] #pci = [''0d:00.0''] pae = 1 apic = 1 vif = [''type=netfront,bridge=priv1'', ''type=netfront,bridge=net1'', ''type=netfront,bridge=net2'', ''type=netfront,bridge=net3'', ''type=netfront,bridge=net4''] serial = ''pty'' disk = [''file:/path/System.img,xvda,w'', ''file:/path/u01.img,xvdb,w'', ''file:/path/swap.img,xvdc,w''] acpi = 1 Crash info from serial console: (XEN) domctl.c:1065:d0 ioport_map:remove f_gport=c100 f_mport=9000 np=100 (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3180 mfn=df03c nr_mfns=4 (XEN) p2m.c:2797:d0 clear_mmio_p2m_entry: gfn_to_mfn failed! gfn=000f3182 (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3100 mfn=df040 nr_mfns=40 ------------[ cut here ]------------ kernel BUG at drivers/pci/access.c:423! invalid opcode: 0000 [#1] SMP CPU 18 Modules linked in: xt_physdev iptable_filter ip_tables xen_blkback xen_netback xen_pciback xen_gntdev xen_evtchn ipmi_devintf ipmi_si lockd sunrpc bridge stp llc bonding dm_round_robin dm_multipath be2iscsi iscsi_boot_sysfs ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic uio ipv6 cxgb3i libcxgbi cxgb3 libiscsi_tcp libiscsi scsi_transport_iscsi xenfs xen_privcmd dm_mirror video sbs sbshc acpi_memhotplug acpi_ipmi ipmi_msghandler parport_pc lp parport igb ses enclosure ixgbe mdio e1000e snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore i2c_i801 snd_page_alloc ghes iTCO_wdt i2c_core pcspkr iTCO_vendor_support i7core_edac ioatdma edac_core hed dca dm_region_hash dm_log dm_mod usb_storage s hpchp sg ahci libahci sd_mod crc_t10dif raid1 ext3 jbd mbcache [last unloaded: raid_class] Pid: 20399, comm: xl Not tainted 2.6.39-200.1.15.el5uek #1 ORACLE CORPORATION SUN FIRE X4370 M2 SERVER /ASSY,SC_BD,T4 RIP: e030:[<ffffffff812704bf>] [<ffffffff812704bf>] pci_block_user_cfg_access+0x4f/0x60 RSP: e02b:ffff88006e609dd8 EFLAGS: 00010202 RAX: 0000000000000024 RBX: 0000000000000001 RCX: 00000000800d0004 RDX: 0000000000000400 RSI: 0000000000000200 RDI: 0000000000000200 RBP: ffff88006e609de8 R08: 0000000000000002 R09: 0000000000000400 R10: 0000000000000000 R11: 0000000000000202 R12: ffff8800731e4000 R13: ffff8800731e4000 R14: ffffffff81565e20 R15: ffff8800731e40a0 FS: 00007fb7b5dfa730(0000) GS:ffff88007f66e000(0000) knlGS:0000000000000000 CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000003340613000 CR3: 0000000068440000 CR4: 0000000000002660 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process xl (pid: 20399, threadinfo ffff88006e608000, task ffff880071fe0500) Stack: 0000000000000000 0000000000000000 ffff88006e609e18 ffffffff812749dc ffff8800731e4000 0000000000000000 ffff8800731e4000 00000000ffffffed ffff88006e609e38 ffffffff812764c4 ffff8800731e4090 0000000000000001 Call Trace: [<ffffffff812749dc>] pci_dev_reset+0x6c/0xd0 [<ffffffff812764c4>] pci_reset_function+0x54/0x70 [<ffffffff8127a6ba>] reset_store+0x5a/0x70 [<ffffffff81342070>] dev_attr_store+0x20/0x30 [<ffffffff811db48f>] flush_write_buffer+0x5f/0x90 [<ffffffff811db856>] sysfs_write_file+0x66/0xa0 [<ffffffff8116dd8e>] vfs_write+0xce/0x190 [<ffffffff8116e3e5>] sys_write+0x55/0x90 [<ffffffff8150a242>] system_call_fastpath+0x16/0x1b Code: 07 00 00 48 c7 c7 e0 73 d7 81 89 c3 83 c8 20 c0 eb 05 41 88 84 24 80 07 00 00 83 e3 01 e8 6a 17 29 00 85 db 75 05 5b 41 5c c9 c3 <0f> 0b eb fe 66 0f 1f 44 00 00 0f 1f 80 00 00 00 00 55 48 89 e5 RIP [<ffffffff812704bf>] pci_block_user_cfg_access+0x4f/0x60 RSP <ffff88006e609dd8> pciback 0000:0d: BTW: xm-create work fine but xl-create trigger the panic as below: On 02/17/13 12:04, Joe Jin wrote:> I got serial console output now and mostly like panic happened by "xl create": > > (XEN) HVM1: Detected Xen v4.1.2-OVM > (XEN) HVM1: CPU speed is 2926 MHz > (XEN) HVM1: Xenbus rings @0xfeffc000, event channel 13 > (XEN) irq.c:264: Dom1 PCI link 0 changed 0 -> 5 > (XEN) HVM1: PCI-ISA link 0 routed to IRQ5 > (XEN) irq.c:264: Dom1 PCI link 1 changed 0 -> 10 > (XEN) HVM1: PCI-ISA link 1 routed to IRQ10 > (XEN) irq.c:264: Dom1 PCI link 2 changed 0 -> 11 > (XEN) HVM1: PCI-ISA link 2 routed to IRQ11 > (XEN) irq.c:264: Dom1 PCI link 3 changed 0 -> 5 > (XEN) HVM1: PCI-ISA link 3 routed to IRQ5 > (XEN) HVM1: pci dev 01:3 INTA->IRQ10 > (XEN) HVM1: pci dev 03:0 INTA->IRQ5 > (XEN) HVM1: pci dev 04:0 INTA->IRQ5 > (XEN) HVM1: pci dev 05:0 INTA->IRQ10 > (XEN) HVM1: pci dev 02:0 bar 10 size 02000000: f0000008 > (XEN) HVM1: pci dev 03:0 bar 14 size 01000000: f2000008 > (XEN) HVM1: pci dev 04:0 bar 30 size 00080000: f3000000 > (XEN) HVM1: pci dev 05:0 bar 30 size 00080000: f3080000 > (XEN) domctl.c:985:d0 memory_map:add: gfn=f3100 mfn=df040 nr_mfns=40 > (XEN) HVM1: pci dev 04:0 bar 1c size 00040000: f3100004 > (XEN) domctl.c:985:d0 memory_map:add: gfn=f3140 mfn=df640 nr_mfns=40 > (XEN) HVM1: pci dev 05:0 bar 1c size 00040000: f3140004 > (XEN) domctl.c:985:d0 memory_map:add: gfn=f3180 mfn=df03c nr_mfns=4 > (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3182 mfn=df03e nr_mfns=1 > (XEN) HVM1: pci dev 04:0 bar 14 size 00004000: f3180004 > (XEN) domctl.c:985:d0 memory_map:add: gfn=f3184 mfn=df63c nr_mfns=4 > (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3186 mfn=df63e nr_mfns=1 > (XEN) HVM1: pci dev 05:0 bar 14 size 00004000: f3184004 > (XEN) HVM1: pci dev 02:0 bar 14 size 00001000: f3188000 > (XEN) HVM1: pci dev 03:0 bar 10 size 00000100: 0000c001 > (XEN) HVM1: pci dev 04:0 bar 10 size 00000100: 0000c101 > (XEN) domctl.c:1041:d0 ioport_map:add f_gport=c100 f_mport=9000 np=100 > (XEN) HVM1: pci dev 05:0 bar 10 size 00000100: 0000c201 > (XEN) domctl.c:1041:d0 ioport_map:add f_gport=c200 f_mport=d000 np=100 > (XEN) HVM1: pci dev 01:1 bar 20 size 00000010: 0000c301 > (XEN) HVM1: Multiprocessor initialisation: > (XEN) HVM1: - CPU0 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. > (XEN) HVM1: - CPU1 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. > (XEN) HVM1: - CPU2 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. > (XEN) HVM1: - CPU3 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. > (XEN) HVM1: - CPU4 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. > (XEN) HVM1: - CPU5 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. > (XEN) HVM1: - CPU6 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. > (XEN) HVM1: - CPU7 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. > (XEN) HVM1: - CPU8 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. > (XEN) HVM1: - CPU9 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. > (XEN) HVM1: - CPU10 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. > (XEN) HVM1: - CPU11 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. > (XEN) HVM1: Writing SMBIOS tables ... > (XEN) HVM1: Loading ROMBIOS ... > (XEN) HVM1: 10364 bytes of ROMBIOS high-memory extensions: > (XEN) HVM1: Relocating to 0xfc000000-0xfc00287c ... done > (XEN) HVM1: Creating MP tables ... > (XEN) HVM1: Loading Cirrus VGABIOS ... > (XEN) domctl.c:985:d0 memory_map:add: gfn=f3000 mfn=df080 nr_mfns=80 > (XEN) HVM1: Loading PCI Option ROM ... > (XEN) HVM1: - Manufacturer: LSI Corporation > (XEN) HVM1: - Product name: LSI MPI Boot Support > (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3000 mfn=df080 nr_mfns=80 > (XEN) domctl.c:985:d0 memory_map:add: gfn=f3080 mfn=df680 nr_mfns=80 > (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3080 mfn=df680 nr_mfns=80 > (XEN) HVM1: Loading ACPI ... > (XEN) HVM1: - Lo data: 000ea020-000ea04f > (XEN) HVM1: - Hi data: fc002c00-fc00eb4f > (XEN) HVM1: vm86 TSS at fc00ec00 > (XEN) HVM1: BIOS map: > (XEN) HVM1: c0000-c8fff: VGA BIOS > (XEN) HVM1: c9000-d4fff: PCI Option ROMs > (XEN) HVM1: eb000-eb302: SMBIOS tables > (XEN) HVM1: f0000-fffff: Main BIOS > (XEN) HVM1: E820 table: > (XEN) HVM1: [00]: 00000000:00000000 - 00000000:0009e000: RAM > (XEN) HVM1: [01]: 00000000:0009e000 - 00000000:0009fc00: RESERVED > (XEN) HVM1: [02]: 00000000:0009fc00 - 00000000:000a0000: RESERVED > (XEN) HVM1: HOLE: 00000000:000a0000 - 00000000:000e0000 > (XEN) HVM1: [03]: 00000000:000e0000 - 00000000:00100000: RESERVED > (XEN) HVM1: [04]: 00000000:00100000 - 00000000:f0000000: RAM > (XEN) HVM1: HOLE: 00000000:f0000000 - 00000000:fc000000 > (XEN) HVM1: [05]: 00000000:fc000000 - 00000001:00000000: RESERVED > (XEN) HVM1: [06]: 00000001:00000000 - 00000004:0f800000: RAM > (XEN) HVM1: Invoking ROMBIOS ... > (XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $ > (XEN) stdvga.c:147:d1 entering stdvga and caching modes > (XEN) HVM1: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert Exp $ > (XEN) HVM1: Bochs BIOS - build: 06/23/99 > (XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $ > (XEN) HVM1: Options: apmbios pcibios eltorito PMM > (XEN) HVM1: > (XEN) HVM1: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 > (XEN) HVM1: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (60000 MBytes) > (XEN) HVM1: ata0-1: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 > (XEN) HVM1: ata0 slave: QEMU HARDDISK ATA-7 Hard-Disk ( 97 GBytes) > (XEN) HVM1: ata1-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 > (XEN) HVM1: ata1 master: QEMU HARDDISK ATA-7 Hard-Disk (20000 MBytes) > (XEN) HVM1: IDE time out > (XEN) HVM1: > (XEN) HVM1: PCI device 1000:0070 not found at index 0 > (XEN) HVM1: PCI device 1000:0072 not found at index 2 > (XEN) HVM1: PCI device 1000:0074 not found at index 0 > (XEN) HVM1: PCI device 1000:0076 not found at index 0 > (XEN) HVM1: PCI device 1000:0077 not found at index 0 > (XEN) HVM1: PCI device 1000:0064 not found at index 0 > (XEN) HVM1: PCI device 1000:0065 not found at index 0 > (XEN) HVM1: PCI device 1000:0080 not found at index 0 > (XEN) HVM1: PCI device 1000:0081 not found at index 0 > (XEN) HVM1: PCI device 1000:0082 not found at index 0 > (XEN) HVM1: PCI device 1000:0083 not found at index 0 > (XEN) HVM1: PCI device 1000:0084 not found at index 0 > (XEN) HVM1: PCI device 1000:0085 not found at index 0 > (XEN) HVM1: PCI device 1000:0086 not found at index 0 > (XEN) HVM1: PCI device 1000:0087 not found at index 0 > (XEN) HVM1: PCI device 1000:0070 not found at index 0 > (XEN) HVM1: PCI device 1000:0072 not found at index 2 > (XEN) HVM1: PCI device 1000:0074 not found at index 0 > (XEN) HVM1: PCI device 1000:0076 not found at index 0 > (XEN) HVM1: PCI device 1000:0077 not found at index 0 > (XEN) HVM1: PCI device 1000:0064 not found at index 0 > (XEN) HVM1: PCI device 1000:0065 not found at index 0 > (XEN) HVM1: PCI device 1000:0080 not found at index 0 > (XEN) HVM1: PCI device 1000:0081 not found at index 0 > (XEN) HVM1: PCI device 1000:0082 not found at index 0 > (XEN) HVM1: PCI device 1000:0083 not found at index 0 > (XEN) HVM1: PCI device 1000:0084 not found at index 0 > (XEN) HVM1: PCI device 1000:0085 not found at index 0 > (XEN) HVM1: PCI device 1000:0086 not found at index 0 > (XEN) HVM1: PCI device 1000:0087 not found at index 0 > (XEN) HVM1: > (XEN) HVM1: > (XEN) HVM1: Press F12 for boot menu. > (XEN) HVM1: > (XEN) HVM1: Booting from Hard Disk... > (XEN) HVM1: Booting from 0000:7c00 > (XEN) HVM1: *** int 15h function AX=00c0, BX=0000 not yet supported! > (XEN) HVM1: *** int 15h function AX=ec00, BX=0002 not yet supported! > (XEN) HVM1: KBD: unsupported int 16h function 03 > (XEN) HVM1: *** int 15h function AX=e980, BX=0000 not yet supported! > (XEN) irq.c:330: Dom1 callback via changed to Direct Vector 0xe9 > (XEN) domctl.c:1065:d0 ioport_map:remove f_gport=c100 f_mport=9000 np=100 > (XEN) domctl.c:1041:d0 ioport_map:add f_gport=c100 f_mport=9000 np=100 > (XEN) domctl.c:1065:d0 ioport_map:remove f_gport=c200 f_mport=d000 np=100 > (XEN) domctl.c:1041:d0 ioport_map:add f_gport=c200 f_mport=d000 np=100 > (XEN) irq.c:264: Dom1 PCI link 0 changed 5 -> 0 > (XEN) irq.c:264: Dom1 PCI link 1 changed 10 -> 0 > (XEN) irq.c:264: Dom1 PCI link 2 changed 11 -> 0 > (XEN) irq.c:264: Dom1 PCI link 3 changed 5 -> 0 > ------------[ cut here ]------------ > WARNING: at fs/proc/generic.c:850 remove_proc_entry+0x22d/0x240() > Hardware name: SUN FIRE X4370 M2 SERVER > remove_proc_entry: removing non-empty directory ''irq/536'', leaking at least ''eth7'' > Modules linked in: xt_physdev iptable_filter ip_tables xen_blkback xen_netback xen_pciback xen_gntdev xen_evtchn ipmi_devintf ipmi_si lockd sunrpc bridge stp llc bonding dm_round_robin dm_multipath be2iscsi iscsi_boot_sysfs ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic uio ipv6 cxgb3i libcxgbi cxgb3 libiscsi_tcp libiscsi scsi_transport_iscsi xenfs xen_privcmd dm_mirror video sbs sbshc acpi_memhotplug acpi_ipmi ipmi_msghandler parport_pc lp parport ses enclosure ixgbe igb e1000e mdio snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc i2c_i801 i2c_core i7core_edac iTCO_wdt pcspkr ioatdma iTCO_vendor_support edac_core dca ghes hed dm_region_hash dm_log dm_mod usb_storageshpchp sg ahci libahci sd_mod crc_t10dif raid1 ext3 jbd mbcache [last unloaded: raid_class]> Pid: 19329, comm: qemu-dm Not tainted 2.6.39-200.1.15.el5uek #1 > Call Trace: > [<ffffffff811d10dd>] ? remove_proc_entry+0x22d/0x240 > [<ffffffff8106de10>] warn_slowpath_common+0x90/0xc0 > [<ffffffff8106df3e>] warn_slowpath_fmt+0x6e/0x70 > [<ffffffff8100a7cf>] ? xen_restore_fl_direct_reloc+0x4/0x4 > [<ffffffff8125cf98>] ? sprintf+0x68/0x70 > [<ffffffff811cfef1>] ? __xlate_proc_name+0x41/0xc0 > [<ffffffff81501b9e>] ? _raw_spin_lock+0xe/0x20 > [<ffffffff811d10dd>] remove_proc_entry+0x22d/0x240 > [<ffffffff8113232c>] ? zap_pte_range+0x1ec/0x400 > [<ffffffff810d9430>] unregister_irq_proc+0xd0/0xf0 > [<ffffffff810d571a>] free_desc+0x2a/0x70 > [<ffffffff810d579e>] irq_free_descs+0x3e/0x80 > [<ffffffff812f7822>] xen_free_irq+0x52/0x70 > [<ffffffff812f82cb>] unbind_from_irq+0xeb/0x180 > [<ffffffff81009f0d>] ? xen_force_evtchn_callback+0xd/0x10 > [<ffffffff812f837f>] evtchn_put+0x1f/0x40 > [<ffffffffa055b4e9>] gntdev_put_map+0x39/0x110 [xen_gntdev] > [<ffffffff812640e1>] ? list_del+0x11/0x40 > [<ffffffffa055c38c>] gntdev_ioctl_unmap_grant_ref+0xac/0xd0 [xen_gntdev] > [<ffffffffa055c708>] gntdev_ioctl+0x98/0xa0 [xen_gntdev] > [<ffffffff8117eb8d>] vfs_ioctl+0x1d/0x50 > [<ffffffff8117f623>] do_vfs_ioctl+0x63/0x1b0 > [<ffffffff8113a740>] ? do_munmap+0x240/0x280 > [<ffffffff8117f804>] sys_ioctl+0x94/0xa0 > [<ffffffff8150a242>] system_call_fastpath+0x16/0x1b > ---[ end trace fa61ba514bac93c7 ]--- > BUG: unable to handle kernel NULL pointer dereference at 000000000000001c > IP: [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40 > PGD 6a2cb067 PUD 71d8c067 PMD 0 > Oops: 0000 [#1] SMP > CPU 0 > Modules linked in: xt_physdev iptable_filter ip_tables xen_blkback xen_netback xen_pciback xen_gntdev xen_evtchn ipmi_devintf ipmi_si lockd sunrpc bridge stp llc bonding dm_round_robin dm_multipath be2iscsi iscsi_boot_sysfs ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic uio ipv6 cxgb3i libcxgbi cxgb3 libiscsi_tcp libiscsi scsi_transport_iscsi xenfs xen_privcmd dm_mirror video sbs sbshc acpi_memhotplug acpi_ipmi ipmi_msghandler parport_pc lp parport ses enclosure ixgbe igb e1000e mdio snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc i2c_i801 i2c_core i7core_edac iTCO_wdt pcspkr ioatdma iTCO_vendor_support edac_core dca ghes hed dm_region_hash dm_log dm_mod usb_storageshpchp sg ahci libahci sd_mod crc_t10dif raid1 ext3 jbd mbcache [last unloaded: raid_class]> > Pid: 19329, comm: qemu-dm Tainted: G W 2.6.39-200.1.15.el5uek #1 ORACLE CORPORATION SUN FIRE X4370 M2 SERVER /ASSY,SC_BD,T4 > RIP: e030:[<ffffffff812f6a76>] [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40 > RSP: e02b:ffff88006cfc5dd8 EFLAGS: 00010246 > RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffff88007e38be48 > RDX: 0000000000000000 RSI: 0000000000000218 RDI: ffff88007e38bd70 > RBP: ffff88006cfc5dd8 R08: 0000000000000001 R09: 0000000000000000 > R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000218 > R13: 0000000000000000 R14: 0000000000000017 R15: 0000000000104701 > FS: 00007ff36f4046e0(0000) GS:ffff88007f49a000(0000) knlGS:0000000000000000 > CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b > CR2: 000000000000001c CR3: 000000005f445000 CR4: 0000000000002660 > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 > DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 > Process qemu-dm (pid: 19329, threadinfo ffff88006cfc4000, task ffff880067084200) > Stack: > ffff88006cfc5e38 ffffffff812f81fc 00007ff36e620000 0000003e38e10000 > ffff88006cfc5e38 000000000000014f 0000000000000206 ffff88006b2e8138 > ffffffff81009f0d ffff88006b951cc0 ffff88006b951cc0 0000000000000000 > Call Trace: > [<ffffffff812f81fc>] unbind_from_irq+0x1c/0x180 > [<ffffffff81009f0d>] ? xen_force_evtchn_callback+0xd/0x10 > [<ffffffff812f837f>] evtchn_put+0x1f/0x40 > [<ffffffffa055b4e9>] gntdev_put_map+0x39/0x110 [xen_gntdev] > [<ffffffff812640e1>] ? list_del+0x11/0x40 > [<ffffffffa055c38c>] gntdev_ioctl_unmap_grant_ref+0xac/0xd0 [xen_gntdev] > [<ffffffffa055c708>] gntdev_ioctl+0x98/0xa0 [xen_gntdev] > [<ffffffff8117eb8d>] vfs_ioctl+0x1d/0x50 > [<ffffffff8117f623>] do_vfs_ioctl+0x63/0x1b0 > [<ffffffff8113a740>] ? do_munmap+0x240/0x280 > [<ffffffff8117f804>] sys_ioctl+0x94/0xa0 > [<ffffffff8150a242>] system_call_fastpath+0x16/0x1b > Code: 8b 64 24 08 c9 c3 0f 1f 80 00 00 00 00 0f 1f 84 00 00 00 00 00 55 48 89 e5 66 66 66 66 90 39 3d f1 16 4c 00 76 0b e8 ea fa ff ff <0f> b7 40 1c c9 c3 89 f9 31 c0 48 c7 c2 2f 72 70 81 be d1 00 00 > RIP [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40 > RSP <ffff88006cfc5dd8> > CR2: 000000000000001c > > Thanks, > Joe > On 02/05/13 17:42, Ian Campbell wrote: >> On Tue, 2013-02-05 at 06:47 +0000, Joe Jin wrote: >>> execute xl-destroy twice crashed my server! >> >> Can you give more details please. >> >> Ian. >> >> >>
On Sat, 2013-02-16 at 05:38 +0000, Joe Jin wrote:> On 02/05/13 17:42, Ian Campbell wrote: > > On Tue, 2013-02-05 at 06:47 +0000, Joe Jin wrote: > >> > execute xl-destroy twice crashed my server! > > Can you give more details please. > > > Hi Ian, > > Sorry the server not available for test now.OK, please let us know if/when you can gather details. Oh I see you already have, thanks!> Would you please help to review my patch for xm-destroy?xend is deprecated and I''m afraid I don''t have many spare cycles to spend on it. I think really it needs to be down to those who want to keep xend alive to review each others patches. Ian.
On Sun, 2013-02-17 at 05:07 +0000, Joe Jin wrote:> Execute "xl-destroy" twice lead the dom0 panic: > config file as below:Thanks.> kernel BUG at drivers/pci/access.c:423![...]> BTW: xm-create work fine but xl-create trigger the panic as below:Both the panic above and the warn+panic below seem like kernel bugs which are simply exposed by xl rather than xl bugs as such. Presumably xl just tickles things different to xm. The dom0 kernel you are using is a bit dated and appears (from the revision) to be rather heavily patched.> > WARNING: at fs/proc/generic.c:850 remove_proc_entry+0x22d/0x240() > > Hardware name: SUN FIRE X4370 M2 SERVER > > remove_proc_entry: removing non-empty directory ''irq/536'', leaking at least ''eth7''[..]> > BUG: unable to handle kernel NULL pointer dereference at 000000000000001c > > IP: [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40Wei recently posted a bug fix for an issue which looked a bit like this last one. That was something do with error handling on allocation failure though so I doubt it represents the actual root cause though. Ian.
Konrad Rzeszutek Wilk
2013-Mar-19 17:41 UTC
Re: [PATCH] xend: disallow multiple destroy() call
On Mon, Feb 04, 2013 at 09:54:23AM +0800, Joe Jin wrote:> Current Xend allowing multiple call destroy() for same domain, this lead > multiple hard resets(FLR) for pci pass-through, and some controller might > failed. > > In our test, we pass through 2 LSI HAB controllers to the PVHVM guest, after > guest brought up, call xm-destroy twice, the adapters''s BIOS will hung, and > we had to reboot the server to recovery it. > > Signed-off-by: Joe Jin <joe.jin@oracle.com> > Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> The other bugs (the kernel ones) are a nice find and should also be fixed - but I think with the upstream kernel they have been mostly fixed.> Cc: Ian Campbell <ian.campbell@citrix.com> > --- > tools/python/xen/xend/XendDomainInfo.py | 11 +++++++++++ > 1 file changed, 11 insertions(+) > > diff --git a/tools/python/xen/xend/XendDomainInfo.py b/tools/python/xen/xend/XendDomainInfo.py > index e9d3e7e..658c3b6 100644 > --- a/tools/python/xen/xend/XendDomainInfo.py > +++ b/tools/python/xen/xend/XendDomainInfo.py > @@ -367,6 +367,8 @@ class XendDomainInfo: > @type refresh_shutdown_lock: threading.Condition > @ivar _deviceControllers: device controller cache for this domain > @type _deviceControllers: dict ''string'' to DevControllers > + @ivar destroying: Is this domain destroying > + @type destroying: bool > """ > > def __init__(self, info, domid = None, dompath = None, augment = False, > @@ -455,6 +457,8 @@ class XendDomainInfo: > self._checkName(self.info[''name_label'']) > > self.metrics = XendVMMetrics(uuid.createString(), self) > + > + self.destroying = False > > > # > @@ -3073,6 +3077,12 @@ class XendDomainInfo: > > if self.domid is None: > return > + > + if self.destroying == False: > + self.destroying = True > + else: > + raise VmError("Domain (domid=%s) is destroying, please wait!", str(self.domid)) > + > from xen.xend import XendDomain > log.debug("XendDomainInfo.destroy: domid=%s", str(self.domid)) > > @@ -3088,6 +3098,7 @@ class XendDomainInfo: > self.info[state] = 0 > self._stateSet(DOM_STATE_HALTED) > except: > + self.destroying = False > log.exception("XendDomainInfo.destroy: domain destruction failed.") > > XendDomain.instance().remove_domain(self) > -- > 1.8.1 > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel >