I was trying to run Xen on an 8-way ES7000 x86_64 system which is basically two 4-way cells put together. I found that Dom0 crashed (see below) while trying to setup devices on the second cell. I found the reason for this is that the patch for mp_register_gsi() is not present in Dom0 so the system runs out of IRQs. But when Dom0 is upgraded to 2.6.13 it should automatically include this. Here is the patch in LK. http://www.kernel.org/diff/diffview.cgi?file=%2Fpub%2Flinux%2Fkernel%2Fv 2.6%2Fpatch-2.6.13.bz2;z=594 Here is the link to the discussion (search for "Let''s not waste"): http://www.ussg.iu.edu/hypermail/linux/kernel/0505.2/index.html#1200 So we will be alright in x86_64 land once the upgrade to 2.6.13 happens. But there is still a problem in x86_32 land. The function pointer for ioapic_renumber_irq() which is called by mp_register_gsi() is initialized only on the ES7000. Dom0 is oblivious to the fact that it is on the ES7000. How can we work around this problem? Aravindh ES7000 x86_64 crash ------------------- ACPI: PCI Interrupt 0000:18:01.1[B] -> GSI 73 (level, low) -> IRQ 73 e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection (XEN) AP: apicid: 26 address: 17 val: 1000000 (XEN) AP: apicid: 26 address: 16 val: 1a9e4 (XEN) AP: rte.vector: 228 irq: -1 =============>(XEN) AP: irq: -1 entry->next: 0 (XEN) CPU: 0 (XEN) EIP: e010:[<ffff83000012820d>] ioapic_guest_write+0x1cd/0x400 (XEN) EFLAGS: 0000000000010286 (XEN) rax: 0000000000000000 rbx: ffff830c001975b4 rcx: 000000000000151b rdx: 000000000000551b (XEN) rsi: 0000000000000001 rdi: 00000000000000df rbp: 00000000ffffffff rsp: ffff830000107e88 (XEN) r8: 0000000000000000 r9: 0000000000004001 r10: 0000000000001500 r11: ffff83000018ec7f (XEN) r12: 000000000000001a r13: ffff8300001975c0 r14: 0000000000000003 r15: 0000000000000000 (XEN) Xen stack trace from rsp=ffff830000107e88: (XEN) 0000000bfffffff4 ffff830000107e98 000000000001a0e4 ffffffffffffffff ffff8800059fbcd8 ffff830000107ed8 ffffffff80584a40 0000000000000001 (XEN) 0000000000000001 ffff830000135cb0 ffffffff00000009 000000100000001a ffff88000001a9e4 ffff8300001ee080 ffff8300001ee080 0000000000000003 (XEN) 0000000000000000 ffff830000145e2a ffffffff80116ebf 0000000000000013 0000000000000001 0000000000000001 ffffffff80584a40 0000000000000000 (XEN) 0000000000000003 00000000000001c8 0000000000000202 0000000000000013 0000000000000000 0000000000000003 0000000000000013 ffffffff80116ebf (XEN) 000000000001a9e4 0000000000000010 ffff8800059fbcd8 0000010000000000 ffffffff80116ebf 000000000000e033 0000000000000202 ffff8800059fbcb8 (XEN) 000000000000e02b 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 ffff8300001ee080 (XEN) Xen call trace from rsp=ffff830000107e88: (XEN) [<ffff830000135cb0>] do_physdev_op+0x1c0/0x310 (XEN) [<ffff830000145e2a>] syscall_enter+0x9a/0xf4 (XEN) (XEN) Pagetable walk from ffff830c001975bc: (XEN) L4 = 0000000000103027 (XEN) L3 = 0000000000000000 (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) CPU0 FATAL PAGE FAULT (XEN) [error_code=0000] (XEN) Faulting linear address: ffff830c001975bc (XEN) **************************************** _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 30 Aug 2005, at 17:42, Puthiyaparambil, Aravindh wrote:> So we will be alright in x86_64 land once the upgrade to 2.6.13 > happens. > But there is still a problem in x86_32 land. The function pointer for > ioapic_renumber_irq() which is called by mp_register_gsi() is > initialized only on the ES7000. Dom0 is oblivious to the fact that it > is > on the ES7000. How can we work around this problem?Why does i386 need that hook, but not x86_64? Is it for legacy ES7000 boxes? -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Puthiyaparambil, Aravindh
2005-Aug-31 02:01 UTC
RE: [Xen-devel] Re: mp_register_gsi() in Dom0
> Why does i386 need that hook, but not x86_64? Is it for legacy ES7000 > boxes?Yes, the hook is for legacy ES7000 systems. It was implemented to accommodate interrupt overrides. We don''t need the hook for our current systems. So it is not of high priority. I think we can ignore it for now if it is a big deal to enable that code in Dom0. The other question I have is whether you are planning on moving to 2.6.13? Or are you going to freeze the kernel level at 2.6.12 and move xen-unstable to xen-testing? Should I be submitting a patch to mp_register_gsi() for 2.6.12? Thanks, Aravindh _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 31 Aug 2005, at 03:01, Puthiyaparambil, Aravindh wrote:> The other question I have is whether you are planning on moving to > 2.6.13? Or are you going to freeze the kernel level at 2.6.12 and move > xen-unstable to xen-testing? Should I be submitting a patch to > mp_register_gsi() for 2.6.12?We''ll certainly move to 2.6.13 before 3.0.0, but we''re not sure if we''ll do so before 3.0-testing. Either way we''ll probably do the port in the next week or so. If you do cook up a patch, we''ll place it in our patches directory, rather than applyinbg directly to the sparse tree. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> The other question I have is whether you are planning on > moving to 2.6.13? Or are you going to freeze the kernel level > at 2.6.12 and move xen-unstable to xen-testing? Should I be > submitting a patch to > mp_register_gsi() for 2.6.12?The plan is to stick on 2.6.12 for the 3.0-testing-r1 CD, but to upgrade to 2.6.13 shortly after. Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Puthiyaparambil, Aravindh
2005-Aug-31 16:37 UTC
RE: [Xen-devel] Re: mp_register_gsi() in Dom0
> If you do cook up a patch, we''ll place it in our patches directory, > rather than applyinbg directly to the sparse tree.Just to make sure that I understand correctly. I submit a patch against "linux-2.6-xen-sparse" but the patch will not be merged. Instead it will be placed in the "patches" director. Or should I be submitting a patch so that in effect a new file gets created in "patches"? Will the patches in the "patches" directory be merged into the Dom0 and DomU kernels during the build process? Thanks Aravindh _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> Just to make sure that I understand correctly. I submit a > patch against "linux-2.6-xen-sparse" but the patch will not > be merged. Instead it will be placed in the "patches" > director. Or should I be submitting a patch so that in effect > a new file gets created in "patches"? > > Will the patches in the "patches" directory be merged into > the Dom0 and DomU kernels during the build process?Yes. Please submit a patch to create a file in patches/linux-2.6.12 Thanks, Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 31 Aug 2005, at 17:37, Puthiyaparambil, Aravindh wrote:> Just to make sure that I understand correctly. I submit a patch against > "linux-2.6-xen-sparse" but the patch will not be merged. Instead it > will > be placed in the "patches" director. Or should I be submitting a patch > so that in effect a new file gets created in "patches"? > > Will the patches in the "patches" directory be merged into the Dom0 and > DomU kernels during the build process?Hmmm, good point. I forgot we already modify those files in our sparse tree. :-) I think really you are best off waiting until we move to 2.6.13. We will probably make a testing release before then, but we will move to 2.6.13 soon after that. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Puthiyaparambil, Aravindh
2005-Aug-31 18:15 UTC
RE: [Xen-devel] Re: mp_register_gsi() in Dom0
I am confused :-). Keir asked me to hold off and wait until the upgrade to 2.6.13 happens.> Hmmm, good point. I forgot we already modify those files in our sparse > tree. :-) I think really you are best off waiting until we move to > 2.6.13. We will probably make a testing release before then, but we > will move to 2.6.13 soon after that. > > -- Keir> > Just to make sure that I understand correctly. I submit a > > patch against "linux-2.6-xen-sparse" but the patch will not > > be merged. Instead it will be placed in the "patches" > > director. Or should I be submitting a patch so that in effect > > a new file gets created in "patches"? > > > > Will the patches in the "patches" directory be merged into > > the Dom0 and DomU kernels during the build process? > > Yes. Please submit a patch to create a file in patches/linux-2.6.12 > > Thanks, > Ian_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 31 Aug 2005, at 19:15, Puthiyaparambil, Aravindh wrote:> I am confused :-). Keir asked me to hold off and wait until the upgrade > to 2.6.13 happens.Crossed wires, and Ian probably forgot that you will be editing files that are in our sparse tree so putting a patch in the patches directory doesn''t work. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Puthiyaparambil, Aravindh
2005-Aug-31 19:11 UTC
RE: [Xen-devel] Re: mp_register_gsi() in Dom0
Keir, I was looking at some of the patches in the "patches" directory. Take i386-cpu-hotplug-updated-for-mm.patch for example. It patches arch/i386/kernel/smpboot.c. This file is also present in linux-2.6-xen-sparse/arch/xen/i386/kernel/ How is this working? Aravindh> -----Original Message----- > From: Keir Fraser [mailto:Keir.Fraser@cl.cam.ac.uk] > Sent: Wednesday, August 31, 2005 2:25 PM > To: Puthiyaparambil, Aravindh > Cc: xen-devel@lists.xensource.com; Koren, Bradley J; > ian.pratt@cl.cam.ac.uk; Ian Pratt; Vessey, Bruce A; Subrahmanian, Raj; > Davis, Jason > Subject: Re: [Xen-devel] Re: mp_register_gsi() in Dom0 > > > On 31 Aug 2005, at 19:15, Puthiyaparambil, Aravindh wrote: > > > I am confused :-). Keir asked me to hold off and wait until theupgrade> > to 2.6.13 happens. > > Crossed wires, and Ian probably forgot that you will be editing files > that are in our sparse tree so putting a patch in the patchesdirectory> doesn''t work. > > -- Keir_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 31 Aug 2005, at 20:11, Puthiyaparambil, Aravindh wrote:> I was looking at some of the patches in the "patches" directory. Take > i386-cpu-hotplug-updated-for-mm.patch for example. It patches > arch/i386/kernel/smpboot.c. This file is also present in > linux-2.6-xen-sparse/arch/xen/i386/kernel/ > > How is this working?The sparse-tree file is also ''patched'' in place. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel