I have a laptop with a USB keyboard and mouse. If I boot it under a standard kernel I can use both keyboards, the USB mouse and the touchpad and this also works under xen 3. However under xen 4 only the USB keyboard and mouse works, not the laptops own keyboard and touchpad. Does anyone know what the problem might be? Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Pasi Kärkkäinen
2010-Jul-21 20:40 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
On Wed, Jul 21, 2010 at 09:27:20PM +0100, M A Young wrote:> I have a laptop with a USB keyboard and mouse. If I boot it under a > standard kernel I can use both keyboards, the USB mouse and the touchpad > and this also works under xen 3. However under xen 4 only the USB > keyboard and mouse works, not the laptops own keyboard and touchpad. Does > anyone know what the problem might be? >Are the built-in keyboard/touchpad ps2 or usb? Does dom0 see all the usb hosts, when using xen4? -- Pasi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wed, 21 Jul 2010, Pasi Kärkkäinen wrote:> On Wed, Jul 21, 2010 at 09:27:20PM +0100, M A Young wrote: >> I have a laptop with a USB keyboard and mouse. If I boot it under a >> standard kernel I can use both keyboards, the USB mouse and the touchpad >> and this also works under xen 3. However under xen 4 only the USB >> keyboard and mouse works, not the laptops own keyboard and touchpad. Does >> anyone know what the problem might be? >> > > Are the built-in keyboard/touchpad ps2 or usb? > Does dom0 see all the usb hosts, when using xen4?The relevant lines in dmesg are (for a non-xen boot) input: Macintosh mouse button emulation as /devices/virtual/input/input3 input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input4 input: PS/2 Mouse as /devices/platform/i8042/serio2/input/input5 input: AlpsPS/2 ALPS GlidePoint as /devices/platform/i8042/serio2/input/input6 so it looks like ps/2. I haven''t noticed any usb devices missing. I have also now checked that the keyboard still doesn''t work even if the USB keyboard is unplugged. Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Pasi Kärkkäinen
2010-Jul-21 21:41 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
On Wed, Jul 21, 2010 at 10:16:58PM +0100, M A Young wrote:> On Wed, 21 Jul 2010, Pasi Kärkkäinen wrote: > >> On Wed, Jul 21, 2010 at 09:27:20PM +0100, M A Young wrote: >>> I have a laptop with a USB keyboard and mouse. If I boot it under a >>> standard kernel I can use both keyboards, the USB mouse and the touchpad >>> and this also works under xen 3. However under xen 4 only the USB >>> keyboard and mouse works, not the laptops own keyboard and touchpad. Does >>> anyone know what the problem might be? >>> >> >> Are the built-in keyboard/touchpad ps2 or usb? >> Does dom0 see all the usb hosts, when using xen4? > > The relevant lines in dmesg are (for a non-xen boot) > > input: Macintosh mouse button emulation as /devices/virtual/input/input3 > input: AT Translated Set 2 keyboard as > /devices/platform/i8042/serio0/input/input4 > input: PS/2 Mouse as /devices/platform/i8042/serio2/input/input5 > input: AlpsPS/2 ALPS GlidePoint as > /devices/platform/i8042/serio2/input/input6 > > so it looks like ps/2. I haven''t noticed any usb devices missing. I have > also now checked that the keyboard still doesn''t work even if the USB > keyboard is unplugged. >Ok. So when you boot dom0 on xen4, do those ps/2 devices get detected? -- Pasi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, 22 Jul 2010, Pasi Kärkkäinen wrote:> On Wed, Jul 21, 2010 at 10:16:58PM +0100, M A Young wrote: >> On Wed, 21 Jul 2010, Pasi Kärkkäinen wrote: >> >>> Does dom0 see all the usb hosts, when using xen4? >> >> The relevant lines in dmesg are (for a non-xen boot) >> >> input: Macintosh mouse button emulation as /devices/virtual/input/input3 >> input: AT Translated Set 2 keyboard as >> /devices/platform/i8042/serio0/input/input4 >> input: PS/2 Mouse as /devices/platform/i8042/serio2/input/input5 >> input: AlpsPS/2 ALPS GlidePoint as >> /devices/platform/i8042/serio2/input/input6 >> >> so it looks like ps/2. I haven''t noticed any usb devices missing. I have >> also now checked that the keyboard still doesn''t work even if the USB >> keyboard is unplugged. >> > > Ok. > > So when you boot dom0 on xen4, do those ps/2 devices get detected?Partially. The relevant input lines (with associated lines that look relevant) under xen are input: Macintosh mouse button emulation as /devices/virtual/input/input3 ... input input3: hash matches pnp 00:08: hash matches ... input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input4 ... input: PS/2 Generic Mouse as /devices/platform/i8042/serio2/input/input9 psmouse.c: Failed to enable mouse on isa0060/serio2 Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Jul-22 17:51 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
> >So when you boot dom0 on xen4, do those ps/2 devices get detected? > > Partially. The relevant input lines (with associated lines that look > relevant) under xen are > > input: Macintosh mouse button emulation as /devices/virtual/input/input3 > ... > input input3: hash matches > pnp 00:08: hash matches > ... > input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input4 > ... > input: PS/2 Generic Mouse as /devices/platform/i8042/serio2/input/input9 > psmouse.c: Failed to enable mouse on isa0060/serio2Looking briefly at the code, it looks to be sending two bytes down the port. What does ''dmesg | grep serio'' show? Something like this: serio: i8042 KBD port at 0x60,0x64 irq 1 serio: i8042 AUX port at 0x60,0x64 irq 12 input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input4 Do you see both ports in /proc/ioports and the IRQ utilized in /proc/interrupts? Are there an interrupts being sent when you type on the keyboard? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, 22 Jul 2010, Konrad Rzeszutek Wilk wrote:> Looking briefly at the code, it looks to be sending two bytes down the > port. What does ''dmesg | grep serio'' show? Something like this: > > serio: i8042 KBD port at 0x60,0x64 irq 1 > serio: i8042 AUX port at 0x60,0x64 irq 12 > input: AT Translated Set 2 keyboard as > /devices/platform/i8042/serio0/input/input4 > > > Do you see both ports in /proc/ioports and the IRQ utilized in > /proc/interrupts? Are there an interrupts being sent when you type on > the keyboard?For a baremetal boot serio: i8042 KBD port at 0x60,0x64 irq 1 serio: i8042 AUX0 port at 0x60,0x64 irq 12 serio: i8042 AUX1 port at 0x60,0x64 irq 12 serio: i8042 AUX2 port at 0x60,0x64 irq 12 serio: i8042 AUX3 port at 0x60,0x64 irq 12 input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input4 input: PS/2 Mouse as /devices/platform/i8042/serio2/input/input5 input: AlpsPS/2 ALPS GlidePoint as /devices/platform/i8042/serio2/input/input6 for the xen boot serio: i8042 KBD port at 0x60,0x64 irq 1 serio: i8042 AUX0 port at 0x60,0x64 irq 12 serio: i8042 AUX1 port at 0x60,0x64 irq 12 serio: i8042 AUX2 port at 0x60,0x64 irq 12 serio: i8042 AUX3 port at 0x60,0x64 irq 12 input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input4 input: PS/2 Generic Mouse as /devices/platform/i8042/serio2/input/input9 psmouse.c: Failed to enable mouse on isa0060/serio2 /proc/ioports is the same for xen and baremetal. /proc/interrupts says 1: 120 38 IO-APIC-edge i8042 12: 609 115 IO-APIC-edge i8042 for baremetal, and for xen 1: 8 0 xen-pirq-ioapic-edge i8042 12: 3 0 xen-pirq-ioapic-edge i8042 but for xen the counts don''t change. Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Jul-23 14:27 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
On Thu, Jul 22, 2010 at 07:54:17PM +0100, M A Young wrote:> On Thu, 22 Jul 2010, Konrad Rzeszutek Wilk wrote: > > >Looking briefly at the code, it looks to be sending two bytes down the > >port. What does ''dmesg | grep serio'' show? Something like this: > > > >serio: i8042 KBD port at 0x60,0x64 irq 1 > >serio: i8042 AUX port at 0x60,0x64 irq 12 > >input: AT Translated Set 2 keyboard as > >/devices/platform/i8042/serio0/input/input4 > > > > > >Do you see both ports in /proc/ioports and the IRQ utilized in > >/proc/interrupts? Are there an interrupts being sent when you type on > >the keyboard? > > For a baremetal boot > serio: i8042 KBD port at 0x60,0x64 irq 1 > serio: i8042 AUX0 port at 0x60,0x64 irq 12 > serio: i8042 AUX1 port at 0x60,0x64 irq 12 > serio: i8042 AUX2 port at 0x60,0x64 irq 12 > serio: i8042 AUX3 port at 0x60,0x64 irq 12 > input: AT Translated Set 2 keyboard as > /devices/platform/i8042/serio0/input/input4 > input: PS/2 Mouse as /devices/platform/i8042/serio2/input/input5 > input: AlpsPS/2 ALPS GlidePoint as > /devices/platform/i8042/serio2/input/input6 > > for the xen boot > > serio: i8042 KBD port at 0x60,0x64 irq 1 > serio: i8042 AUX0 port at 0x60,0x64 irq 12 > serio: i8042 AUX1 port at 0x60,0x64 irq 12 > serio: i8042 AUX2 port at 0x60,0x64 irq 12 > serio: i8042 AUX3 port at 0x60,0x64 irq 12 > input: AT Translated Set 2 keyboard as > /devices/platform/i8042/serio0/input/input4 > input: PS/2 Generic Mouse as /devices/platform/i8042/serio2/input/input9 > psmouse.c: Failed to enable mouse on isa0060/serio2 > > /proc/ioports is the same for xen and baremetal. /proc/interrupts says > 1: 120 38 IO-APIC-edge i8042 > 12: 609 115 IO-APIC-edge i8042 > for baremetal, and for xen > 1: 8 0 xen-pirq-ioapic-edge i8042 > 12: 3 0 xen-pirq-ioapic-edge i8042 > but for xen the counts don''t change.OK. So we need to dig in this a bit further. Can you enable all of the debugging options on Xen and Linux and send the serial log? And also run ''apic=debug'' on Xen command line. Pasi''s PV-OPS Wiki page has a whole section on what to set for a full debug enabled boot. Another thing that might be worth checking is to boot the PV-OPS kernel with a 3.4-testing Xen hypervisor (it actually does boot at least on my SuperMicro X8DBN+- but don''t try to use the 4.x-unstable tool stack with it :-)). That would eliminate the problem being in the Linux kernel. FYI, I don''t have a PS/2 mouse, but I do have a PS/2 keyboard that works OK. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Fri, 23 Jul 2010, Konrad Rzeszutek Wilk wrote:> OK. So we need to dig in this a bit further. Can you enable all of the > debugging options on Xen and Linux and send the serial log? And also > run ''apic=debug'' on Xen command line. Pasi''s PV-OPS Wiki page has a > whole section on what to set for a full debug enabled boot.Unfortunately I don''t have a serial console to output to. The best I can do is the dmesg outputs for xen and the kernel with I am attaching (bzipped).> Another thing that might be worth checking is to boot the PV-OPS kernel > with a 3.4-testing Xen hypervisor (it actually does boot at least on my > SuperMicro X8DBN+- but don''t try to use the 4.x-unstable tool stack with > it :-)).If I change to xen 3.4 but keep the kernel then both keyboards and mice work, so the problem is unlikely to be the kernel. Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Sun, 8 Aug 2010, M A Young wrote:> Unfortunately I don''t have a serial console to output to. The best I can do > is the dmesg outputs for xen and the kernel with I am attaching (bzipped).I found the conring_size setting and got a more complete dmesg log. Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Aug-09 04:42 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
On Sun, Aug 08, 2010 at 09:16:56PM +0100, M A Young wrote:> On Sun, 8 Aug 2010, M A Young wrote: > > >Unfortunately I don''t have a serial console to output to. The best > >I can do is the dmesg outputs for xen and the kernel with I am > >attaching (bzipped). > > I found the conring_size setting and got a more complete dmesg log.Cool. I will take a look at it after the LinuxCon. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Aug-16 15:46 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
On Sun, Aug 08, 2010 at 09:16:56PM +0100, M A Young wrote:> On Sun, 8 Aug 2010, M A Young wrote: > > >Unfortunately I don''t have a serial console to output to. The best > >I can do is the dmesg outputs for xen and the kernel with I am > >attaching (bzipped). > > I found the conring_size setting and got a more complete dmesg log.<sigh> I looked through it and also through your previous kernel and Xen outputs and nothing sparked that ''Aha!'' moment. There are a couple of things we can try: - Compare this with the output from Xen 3.4 and see if the IOAPIC lines are different. Especially if these: (XEN) IOAPIC[0]: Set PCI routing entry (2-12 -> 0x78 -> IRQ 12 Mode:0 Active:0) (XEN) IOAPIC[0]: Set PCI routing entry (2-1 -> 0x28 -> IRQ 1 Mode:0 Active:0 are different. I think that previous to Xen 4, the pv-ops kernel could not set the IOAPIC entries below pin 16, so you would not see them and instead it would have these programmed: (XEN) 01 001 01 0 0 0 0 0 1 1 28 (XEN) 0c 001 01 0 0 0 0 0 1 1 78 Which is OK, as the trigger and polarity look to be correct. - Boot the Xen4, and trigger the IOAPIC debug printout. I think this is can be done via "xm send-keys i". Also the ''q'' output would be usefull (it will tell us which ioports domain 0 has access to - we should see dom0 see 0x60 and 0x64), and irq 1, and 12. - The /proc/interrupts is interesting. It shows no interrupts on your other CPU, but the IOAPIC tells us that the irq''s should be blasted to both CPUs. So I wonder if it got reprogrammed by the ''irqbalance'' daemon. One way to find that out is to do the ''xm send-keys i'' thingie to see if the destination field is the same. - We can also compare this to baremetal IOAPIC programming. It should be the _same_ as what Xen does. What we can do is provide ''apic=debug'' and that will print out the IOAPIC entries of baremetal kernel. The values for irq 1 and 12 ought to be same as what Xen saw and programmed it too. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Mon, 16 Aug 2010, Konrad Rzeszutek Wilk wrote:> There are a couple of things we can try: > - Compare this with the output from Xen 3.4 and see if the IOAPIC lines > are different. Especially if these: > (XEN) IOAPIC[0]: Set PCI routing entry (2-12 -> 0x78 -> IRQ 12 Mode:0 Active:0) > (XEN) IOAPIC[0]: Set PCI routing entry (2-1 -> 0x28 -> IRQ 1 Mode:0 Active:0 > > are different. I think that previous to Xen 4, the pv-ops kernel could > not set the IOAPIC entries below pin 16, so you would not see them and > instead it would have these programmed: > (XEN) 01 001 01 0 0 0 0 0 1 1 28 > (XEN) 0c 001 01 0 0 0 0 0 1 1 78 > > Which is OK, as the trigger and polarity look to be correct.Logs attached as dmesg.xen3 and xm.xen3> - Boot the Xen4, and trigger the IOAPIC debug printout. I think this is > can be done via "xm send-keys i". Also the ''q'' output would be > usefull (it will tell us which ioports domain 0 has access to - we > should see dom0 see 0x60 and 0x64), and irq 1, and 12.attached as xm.debugkeys> - We can also compare this to baremetal IOAPIC programming. It should > be the _same_ as what Xen does. What we can do is provide > ''apic=debug'' and that will print out the IOAPIC entries of baremetal > kernel. The values for irq 1 and 12 ought to be same as what Xen saw > and programmed it too.attached as dmesg.baremetal Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Aug-16 22:33 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
On Mon, Aug 16, 2010 at 10:05:58PM +0100, M A Young wrote:> On Mon, 16 Aug 2010, Konrad Rzeszutek Wilk wrote: > > >There are a couple of things we can try: > >- Compare this with the output from Xen 3.4 and see if the IOAPIC lines > > are different. Especially if these: > >(XEN) IOAPIC[0]: Set PCI routing entry (2-12 -> 0x78 -> IRQ 12 Mode:0 Active:0) > >(XEN) IOAPIC[0]: Set PCI routing entry (2-1 -> 0x28 -> IRQ 1 Mode:0 Active:0 > > > >are different. I think that previous to Xen 4, the pv-ops kernel could > >not set the IOAPIC entries below pin 16, so you would not see them and > >instead it would have these programmed: > >(XEN) 01 001 01 0 0 0 0 0 1 1 28 > >(XEN) 0c 001 01 0 0 0 0 0 1 1 78 > > > >Which is OK, as the trigger and polarity look to be correct. > > Logs attached as dmesg.xen3 and xm.xen3Cool. So this is what I see (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: | (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: (XEN) 00 000 00 1 0 0 0 0 0 0 00 (XEN) 00 000 00 1 0 0 0 0 0 0 00 (XEN) 01 001 01 0 0 0 0 0 1 1 20 | (XEN) 01 001 01 0 0 0 0 0 1 1 28 (XEN) 02 001 01 0 0 0 0 0 1 1 F0 (XEN) 02 001 01 0 0 0 0 0 1 1 F0 (XEN) 03 001 01 0 0 0 0 0 1 1 28 | (XEN) 03 001 01 0 0 0 0 0 1 1 30 (XEN) 04 001 01 0 0 0 0 0 1 1 30 | (XEN) 04 001 01 0 0 0 0 0 1 1 38 (XEN) 05 001 01 0 0 0 0 0 1 1 38 | (XEN) 05 001 01 0 0 0 0 0 1 1 40 (XEN) 06 001 01 0 0 0 0 0 1 1 40 | (XEN) 06 001 01 0 0 0 0 0 1 1 48 (XEN) 07 001 01 0 0 0 0 0 1 1 48 | (XEN) 07 001 01 0 0 0 0 0 1 1 50 (XEN) 08 001 01 0 0 0 0 0 1 1 50 | (XEN) 08 001 01 0 0 0 0 0 1 1 58 (XEN) 09 001 01 1 1 0 0 0 1 1 58 | (XEN) 09 001 01 1 1 0 0 0 1 1 60 (XEN) 0a 001 01 0 0 0 0 0 1 1 60 | (XEN) 0a 001 01 0 0 0 0 0 1 1 68 (XEN) 0b 001 01 0 0 0 0 0 1 1 68 | (XEN) 0b 001 01 0 0 0 0 0 1 1 70 (XEN) 0c 001 01 0 0 0 0 0 1 1 70 | (XEN) 0c 001 01 0 0 0 0 0 1 1 78 (XEN) 0d 001 01 0 0 0 0 0 1 1 78 | (XEN) 0d 001 01 0 0 0 0 0 1 1 88 (XEN) 0e 001 01 0 0 0 0 0 1 1 88 | (XEN) 0e 001 01 0 0 0 0 0 1 1 90 (XEN) 0f 001 01 0 0 0 0 0 1 1 90 | (XEN) 0f 001 01 0 0 0 0 0 1 1 98 (XEN) 10 000 00 1 0 0 0 0 0 0 00 (XEN) 10 000 00 1 0 0 0 0 0 0 00 .. Left column is 3.4, right is 4.0. The one thing that is odd is that in 4.0 we start with vector 0x28 while in 3.4 it is with 0x20. It looks as if one vector is getten eaten. But that should not be such an issue as the internal mapping of vector->irq is still proper...> > >- Boot the Xen4, and trigger the IOAPIC debug printout. I think this is > > can be done via "xm send-keys i". Also the ''q'' output would be > > usefull (it will tell us which ioports domain 0 has access to - we > > should see dom0 see 0x60 and 0x64), and irq 1, and 12. > > attached as xm.debugkeysOK. The ioports are OK, the vectors and irq are fine too.> > >- We can also compare this to baremetal IOAPIC programming. It should > > be the _same_ as what Xen does. What we can do is provide > > ''apic=debug'' and that will print out the IOAPIC entries of baremetal > > kernel. The values for irq 1 and 12 ought to be same as what Xen saw > > and programmed it too. > > attached as dmesg.baremetalThanks. It seems to do the thing that baremetal always does.. <sigh> Still no idea what is the trouble with your box. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Aug-18 23:25 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
On Mon, Aug 16, 2010 at 06:33:54PM -0400, Konrad Rzeszutek Wilk wrote:> On Mon, Aug 16, 2010 at 10:05:58PM +0100, M A Young wrote: > > On Mon, 16 Aug 2010, Konrad Rzeszutek Wilk wrote: > > > > >There are a couple of things we can try: > > >- Compare this with the output from Xen 3.4 and see if the IOAPIC lines > > > are different. Especially if these: > > >(XEN) IOAPIC[0]: Set PCI routing entry (2-12 -> 0x78 -> IRQ 12 Mode:0 Active:0) > > >(XEN) IOAPIC[0]: Set PCI routing entry (2-1 -> 0x28 -> IRQ 1 Mode:0 Active:0 > > > > > >are different. I think that previous to Xen 4, the pv-ops kernel could > > >not set the IOAPIC entries below pin 16, so you would not see them and > > >instead it would have these programmed: > > >(XEN) 01 001 01 0 0 0 0 0 1 1 28 > > >(XEN) 0c 001 01 0 0 0 0 0 1 1 78 > > > > > >Which is OK, as the trigger and polarity look to be correct. > > > > Logs attached as dmesg.xen3 and xm.xen3 > > Cool. So this is what I see > > (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: | (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: > (XEN) 00 000 00 1 0 0 0 0 0 0 00 (XEN) 00 000 00 1 0 0 0 0 0 0 00 > (XEN) 01 001 01 0 0 0 0 0 1 1 20 | (XEN) 01 001 01 0 0 0 0 0 1 1 28 > (XEN) 02 001 01 0 0 0 0 0 1 1 F0 (XEN) 02 001 01 0 0 0 0 0 1 1 F0 > (XEN) 03 001 01 0 0 0 0 0 1 1 28 | (XEN) 03 001 01 0 0 0 0 0 1 1 30 > (XEN) 04 001 01 0 0 0 0 0 1 1 30 | (XEN) 04 001 01 0 0 0 0 0 1 1 38 > (XEN) 05 001 01 0 0 0 0 0 1 1 38 | (XEN) 05 001 01 0 0 0 0 0 1 1 40 > (XEN) 06 001 01 0 0 0 0 0 1 1 40 | (XEN) 06 001 01 0 0 0 0 0 1 1 48 > (XEN) 07 001 01 0 0 0 0 0 1 1 48 | (XEN) 07 001 01 0 0 0 0 0 1 1 50 > (XEN) 08 001 01 0 0 0 0 0 1 1 50 | (XEN) 08 001 01 0 0 0 0 0 1 1 58 > (XEN) 09 001 01 1 1 0 0 0 1 1 58 | (XEN) 09 001 01 1 1 0 0 0 1 1 60 > (XEN) 0a 001 01 0 0 0 0 0 1 1 60 | (XEN) 0a 001 01 0 0 0 0 0 1 1 68 > (XEN) 0b 001 01 0 0 0 0 0 1 1 68 | (XEN) 0b 001 01 0 0 0 0 0 1 1 70 > (XEN) 0c 001 01 0 0 0 0 0 1 1 70 | (XEN) 0c 001 01 0 0 0 0 0 1 1 78 > (XEN) 0d 001 01 0 0 0 0 0 1 1 78 | (XEN) 0d 001 01 0 0 0 0 0 1 1 88 > (XEN) 0e 001 01 0 0 0 0 0 1 1 88 | (XEN) 0e 001 01 0 0 0 0 0 1 1 90 > (XEN) 0f 001 01 0 0 0 0 0 1 1 90 | (XEN) 0f 001 01 0 0 0 0 0 1 1 98 > (XEN) 10 000 00 1 0 0 0 0 0 0 00 (XEN) 10 000 00 1 0 0 0 0 0 0 00 > .. > Left column is 3.4, right is 4.0. > > The one thing that is odd is that in 4.0 we start with vector 0x28 while > in 3.4 it is with 0x20. > > It looks as if one vector is getten eaten. But that should not be such > an issue as the internal mapping of vector->irq is still proper...This looks like a one-offset bug. On a related note this might be the same as when my irq delivery goes haywire under load and the IRQ delivery is put on another CPU but that whole process looks to be taking 10-20 seconds. When did you start observing this? It is a bit hard to bisect code between 3.4. and 4.0. Did it work with 4.0? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wed, 18 Aug 2010, Konrad Rzeszutek Wilk wrote:> When did you start observing this? It is a bit hard to bisect code > between 3.4. and 4.0. Did it work with 4.0?No, I have just retested it with xen 4.0.0 (plus the two patches to get it to work with the recent 2.6.32.x kernels) and I have the same problem with the laptop keyboard as with the 4.0.1 release candidates. Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Mon, 16 Aug 2010, Konrad Rzeszutek Wilk wrote:> The one thing that is odd is that in 4.0 we start with vector 0x28 while > in 3.4 it is with 0x20. > > It looks as if one vector is getten eaten. But that should not be such > an issue as the internal mapping of vector->irq is still proper...I can tell you why that happens. For 3.4 new vectors are allocated in the order 32,40,...,216 then 33,41,... etc. in assign_irq_vector from xen/arch/x86/irq.c For 4.0 new vectors are allocated in the order 40,48,...,216 then 33,41,... etc. in __assign_irq_vector from xen/arch/x86/irq.c So for 4.0 the allocation of vector 32 is never attempted, but as it is already reserved for IRQ_MOVE_CLEANUP_VECTOR this shouldn''t matter. Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
I have more information. If I boot giving xen the nosmp option then it can see the laptop''s keyboard. If I compare the xm demesg output of boots with and without nosmp (having also run xm debug-keys z) I get the output below. On reflection I think the do_IRQ warnings migth be relevant. The full logs are attached. Michale Young 9c9 < (XEN) Command line: ---> (XEN) Command line: nosmp48c48 < (XEN) Detected 2394.072 MHz processor. ---> (XEN) Detected 2394.108 MHz processor.58d57 < (XEN) Total of 2 processors activated. 61d59 < (XEN) checking TSC synchronization across 2 CPUs: passed. 63c61 < (XEN) Brought up 2 CPUs ---> (XEN) Brought up 1 CPUs68c66 < (XEN) Dom0 alloc.: 0000000110000000->0000000118000000 (967236 pages to be allocated) ---> (XEN) Dom0 alloc.: 0000000110000000->0000000118000000 (967255 pages to be allocated)72c70 < (XEN) Phys-Mach map: ffffffff83b38000->ffffffff842d9220 ---> (XEN) Phys-Mach map: ffffffff83b38000->ffffffff842d92b878c76 < (XEN) Dom0 has maximum 2 VCPUs ---> (XEN) Dom0 has maximum 1 VCPUs86,95d83 < (XEN) do_IRQ: 1.120 No irq handler for vector (irq -1) < (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) < (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) < (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) < (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) < (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) < (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) < (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) < (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) < (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) 123c111 < (XEN) 0d 003 03 1 0 0 0 0 1 1 88 ---> (XEN) 0d 001 01 1 0 0 0 0 1 1 88127c115 < (XEN) 11 003 03 1 1 0 1 0 1 1 A8 ---> (XEN) 11 001 01 1 1 0 1 0 1 1 A8_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Aug-23 14:24 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
On Sat, Aug 21, 2010 at 10:00:37PM +0100, M A Young wrote:> On Mon, 16 Aug 2010, Konrad Rzeszutek Wilk wrote: > > >The one thing that is odd is that in 4.0 we start with vector 0x28 while > >in 3.4 it is with 0x20. > > > >It looks as if one vector is getten eaten. But that should not be such > >an issue as the internal mapping of vector->irq is still proper... > > I can tell you why that happens. > For 3.4 new vectors are allocated in the order 32,40,...,216 then > 33,41,... etc. in assign_irq_vector from xen/arch/x86/irq.c > For 4.0 new vectors are allocated in the order 40,48,...,216 then > 33,41,... etc. in __assign_irq_vector from xen/arch/x86/irq.c > So for 4.0 the allocation of vector 32 is never attempted, but as it > is already reserved for IRQ_MOVE_CLEANUP_VECTOR this shouldn''t > matter.Argh. So red-heering then. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Aug-23 15:18 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
On Sun, Aug 22, 2010 at 09:03:14PM +0100, M A Young wrote:> I have more information. If I boot giving xen the nosmp option then > it can see the laptop''s keyboard. If I compare the xm demesg output > of boots with and without nosmp (having also run xm debug-keys z) I > get the output below. On reflection I think the do_IRQ warningsWhat amchine is this? You mentioned that this a laptop of some kind. Looks to be a Dell Inspiron 9400 aka e1705?> migth be relevant. The full logs are attached.It is the best we have for now.> > Michale Young > > 9c9 > < (XEN) Command line: --- > >(XEN) Command line: nosmp > 48c48 > < (XEN) Detected 2394.072 MHz processor. > --- > >(XEN) Detected 2394.108 MHz processor. > 58d57 > < (XEN) Total of 2 processors activated. > 61d59 > < (XEN) checking TSC synchronization across 2 CPUs: passed. > 63c61 > < (XEN) Brought up 2 CPUs > --- > >(XEN) Brought up 1 CPUs > 68c66 > < (XEN) Dom0 alloc.: 0000000110000000->0000000118000000 (967236 pages to be allocated) > --- > >(XEN) Dom0 alloc.: 0000000110000000->0000000118000000 (967255 pages to be allocated) > 72c70 > < (XEN) Phys-Mach map: ffffffff83b38000->ffffffff842d9220 > --- > >(XEN) Phys-Mach map: ffffffff83b38000->ffffffff842d92b8 > 78c76 > < (XEN) Dom0 has maximum 2 VCPUs > --- > >(XEN) Dom0 has maximum 1 VCPUs > 86,95d83 > < (XEN) do_IRQ: 1.120 No irq handler for vector (irq -1) > < (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1)The 1 is the CPU1 ID, while the second number is the vector :120 (0x78) and vector 140 (0x90). Looking at the IO APIC programming: (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: (XEN) 0c 001 01 0 0 0 0 0 1 1 78 (XEN) 0e 001 01 0 0 0 0 0 1 1 90 So that would IRQ 12 and 14. Considering the problem we have been having is with IRQ 12, this looks to point to a problem. You have the flat mode, which means> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICsfrom "Pentium processor system archicture" book (http://books.google.com/books?id=TVzjEZg1--YC&pg=PA359&lpg=PA359&dq=Logical+Destination+Register&source=bl&ots=iz8zOOx1BY&sig=cP83Oo74cF6dY_EUCmb4o2vdHUs&hl=en&ei=3IdyTLDPCIH98Ab_4a35DA&sa=X&oi=book_result&ct=result&resnum=6&ved=0CC8Q6AEwBQ) that irq 12 and 14 are being both sent to the CPU1 based on decoding the "Log" and "Phy" flags:. We could hack it and make sure that in setup_IO_APIC_irqs for IRQ 12, we make it send to both CPUs. Something like this (compile tested only): diff -r 5218db847b58 xen/arch/x86/io_apic.c --- a/xen/arch/x86/io_apic.c Tue Aug 17 19:32:37 2010 +0100 +++ b/xen/arch/x86/io_apic.c Mon Aug 23 11:03:50 2010 -0400 @@ -1004,6 +1004,13 @@ cfg = irq_cfg(irq); SET_DEST(entry.dest.dest32, entry.dest.logical.logical_dest, cpu_mask_to_apicid(cfg->domain)); + if (irq==12) { + apic_printk(APIC_VERBOSE, " hack enabled IRQ 12(%x) -> %x.\n", + (unsigned int)entry.vector, + cpu_mask_to_apicid(TARGET_CPUS)); + SET_DEST(entry.dest.dest32, entry.dest.logical.logical_dest, + cpu_mask_to_apicid(TARGET_CPUS)); + } spin_lock_irqsave(&ioapic_lock, flags); io_apic_write(apic, 0x11+2*pin, *(((int *)&entry)+1)); io_apic_write(apic, 0x10+2*pin, *(((int *)&entry)+0)); If that works where you have both CPUs, it helps us a bit (perhaps the the cfg->domain cpumask_t isn''t programmed propely?). Interestingly, in 3.4, the majority of the pins (based on your previous emails) are programmed to go to CPU1. This was also the behavior for Xen 4.0.1 and >. The debug output you included, which was 4.00, has some going to all CPUS: (XEN) 02 0FF 0F 0 0 0 0 0 1 1 F0 (XEN) 0d 003 03 1 0 0 0 0 1 1 88 (XEN) 11 003 03 1 1 0 1 0 1 1 A8 which is completly different with how 4.0.1 and 3.4 behaved. The plot thickens!> < (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) > < (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) > < (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) > < (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) > < (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) > < (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) > < (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) > < (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) > 123c111 > < (XEN) 0d 003 03 1 0 0 0 0 1 1 88 > --- > >(XEN) 0d 001 01 1 0 0 0 0 1 1 88 > 127c115 > < (XEN) 11 003 03 1 1 0 1 0 1 1 A8 > --- > >(XEN) 11 001 01 1 1 0 1 0 1 1 A8> __ __ _ _ ___ ___ ____ __ _ _____ > \ \/ /___ _ __ | || | / _ \ / _ \ | ___| / _| ___/ |___ / > \ // _ \ ''_ \ | || |_| | | | | | |_|___ \ | |_ / __| | |_ \ > / \ __/ | | | |__ _| |_| | |_| |__|__) || _| (__| |___) | > /_/\_\___|_| |_| |_|(_)___(_)___/ |____(_)_| \___|_|____/ > > (XEN) Xen version 4.0.0 (michael@home) (gcc version 4.4.4 20100630 (Red Hat 4.4.4-10) (GCC) ) Mon Jul 26 23:05:53 BST 2010 > (XEN) Latest ChangeSet: unavailable > (XEN) Command line: > (XEN) Video information: > (XEN) VGA is text mode 80x25, font 8x16 > (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds > (XEN) Disc information: > (XEN) Found 1 MBR signatures > (XEN) Found 1 EDD information structures > (XEN) Xen-e820 RAM map: > (XEN) 0000000000000000 - 000000000009f000 (usable) > (XEN) 000000000009f000 - 00000000000a0000 (reserved) > (XEN) 0000000000100000 - 00000000df66d800 (usable) > (XEN) 00000000df66d800 - 00000000e0000000 (reserved) > (XEN) 00000000f8000000 - 00000000fc000000 (reserved) > (XEN) 00000000fec00000 - 00000000fec10000 (reserved) > (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) > (XEN) 00000000fed20000 - 00000000fed90000 (reserved) > (XEN) 00000000feda0000 - 00000000feda6000 (reserved) > (XEN) 00000000fee00000 - 00000000fee10000 (reserved) > (XEN) 00000000ffe00000 - 0000000100000000 (reserved) > (XEN) 0000000100000000 - 0000000120000000 (usable) > (XEN) ACPI: RSDP 000FBC60, 0024 (r2 DELL ) > (XEN) ACPI: XSDT DF66F200, 0064 (r1 DELL M08 27D80A10 ASL 61) > (XEN) ACPI: FACP DF66F09C, 00F4 (r4 DELL M08 27D80A10 ASL 61) > (XEN) ACPI: DSDT DF66F800, 5477 (r2 INT430 SYSFexxx 1001 INTL 20050624) > (XEN) ACPI: FACS DF67E000, 0040 > (XEN) ACPI: HPET DF66F300, 0038 (r1 DELL M08 1 ASL 61) > (XEN) ACPI: APIC DF66F400, 0068 (r1 DELL M08 27D80A10 ASL 47) > (XEN) ACPI: MCFG DF66F3C0, 003E (r16 DELL M08 27D80A10 ASL 61) > (XEN) ACPI: SLIC DF66F49C, 0176 (r1 DELL M08 27D80A10 ASL 61) > (XEN) ACPI: OSFR DF66EA00, 002C (r1 DELL M08 27D80A10 ASL 61) > (XEN) ACPI: BOOT DF66EFC0, 0028 (r1 DELL M08 27D80A10 ASL 61) > (XEN) ACPI: SSDT DF66D97E, 04CC (r1 PmRef CpuPm 3000 INTL 20050624) > (XEN) System RAM: 4031MB (4128332kB) > (XEN) Domain heap initialised > (XEN) Processor #0 7:7 APIC version 20 > (XEN) Processor #1 7:7 APIC version 20 > (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 > (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs > (XEN) Using scheduler: SMP Credit Scheduler (credit) > (XEN) Detected 2394.072 MHz processor. > (XEN) Initing memory sharing. > (XEN) VMX: Supported advanced features: > (XEN) - APIC MMIO access virtualisation > (XEN) - APIC TPR shadow > (XEN) - Virtual NMI > (XEN) - MSR direct-access bitmap > (XEN) HVM: ASIDs disabled. > (XEN) HVM: VMX enabled > (XEN) I/O virtualisation disabled > (XEN) Total of 2 processors activated. > (XEN) ENABLING IO-APIC IRQs > (XEN) -> Using new ACK method > (XEN) checking TSC synchronization across 2 CPUs: passed. > (XEN) Platform timer is 14.318MHz HPET > (XEN) Brought up 2 CPUs > (XEN) *** LOADING DOMAIN 0 *** > (XEN) Xen kernel: 64-bit, lsb, compat32 > (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1a6f000 > (XEN) PHYSICAL MEMORY ARRANGEMENT: > (XEN) Dom0 alloc.: 0000000110000000->0000000118000000 (967236 pages to be allocated) > (XEN) VIRTUAL MEMORY ARRANGEMENT: > (XEN) Loaded kernel: ffffffff81000000->ffffffff81a6f000 > (XEN) Init. ramdisk: ffffffff81a6f000->ffffffff83b37e00 > (XEN) Phys-Mach map: ffffffff83b38000->ffffffff842d9220 > (XEN) Start info: ffffffff842da000->ffffffff842da4b4 > (XEN) Page tables: ffffffff842db000->ffffffff84300000 > (XEN) Boot stack: ffffffff84300000->ffffffff84301000 > (XEN) TOTAL: ffffffff80000000->ffffffff84400000 > (XEN) ENTRY ADDRESS: ffffffff81819200 > (XEN) Dom0 has maximum 2 VCPUs > (XEN) Scrubbing Free RAM: .done. > (XEN) Xen trace buffers: disabled > (XEN) Std. Loglevel: Errors and warnings > (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings) > (XEN) Xen is relinquishing VGA console. > (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen) > (XEN) Freed 172kB init memory. > (XEN) do_IRQ: 1.120 No irq handler for vector (irq -1) > (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) > (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) > (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) > (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) > (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) > (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) > (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) > (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) > (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) > (XEN) number of MP IRQ sources: 15. > (XEN) number of IO-APIC #2 registers: 24. > (XEN) testing the IO APIC....................... > (XEN) IO APIC #2...... > (XEN) .... register #00: 00000000 > (XEN) ....... : physical APIC id: 00 > (XEN) ....... : Delivery Type: 0 > (XEN) ....... : LTS : 0 > (XEN) .... register #01: 00170020 > (XEN) ....... : max redirection entries: 0017 > (XEN) ....... : PRQ implemented: 0 > (XEN) ....... : IO APIC version: 0020 > (XEN) .... IRQ redirection table: > (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: > (XEN) 00 000 00 1 0 0 0 0 0 0 00 > (XEN) 01 001 01 0 0 0 0 0 1 1 28 > (XEN) 02 0FF 0F 0 0 0 0 0 1 1 F0 > (XEN) 03 001 01 0 0 0 0 0 1 1 30 > (XEN) 04 001 01 0 0 0 0 0 1 1 38 > (XEN) 05 001 01 0 0 0 0 0 1 1 40 > (XEN) 06 001 01 0 0 0 0 0 1 1 48 > (XEN) 07 001 01 0 0 0 0 0 1 1 50 > (XEN) 08 001 01 0 0 0 0 0 1 1 58 > (XEN) 09 001 01 0 1 0 0 0 1 1 60 > (XEN) 0a 001 01 0 0 0 0 0 1 1 68 > (XEN) 0b 001 01 0 0 0 0 0 1 1 70 > (XEN) 0c 001 01 0 0 0 0 0 1 1 78 > (XEN) 0d 003 03 1 0 0 0 0 1 1 88 > (XEN) 0e 001 01 0 0 0 0 0 1 1 90 > (XEN) 0f 001 01 0 0 0 0 0 1 1 98 > (XEN) 10 001 01 0 1 0 1 0 1 1 A0 > (XEN) 11 003 03 1 1 0 1 0 1 1 A8 > (XEN) 12 001 01 0 1 0 1 0 1 1 31 > (XEN) 13 000 00 1 0 0 0 0 0 0 00 > (XEN) 14 001 01 0 1 0 1 0 1 1 D8 > (XEN) 15 001 01 0 1 0 1 0 1 1 21 > (XEN) 16 001 01 0 1 0 1 0 1 1 D0 > (XEN) 17 000 00 1 0 0 0 0 0 0 00 > (XEN) Using vector-based indexing > (XEN) IRQ to pin mappings: > (XEN) IRQ240 -> 0:2 > (XEN) IRQ40 -> 0:1 > (XEN) IRQ48 -> 0:3 > (XEN) IRQ56 -> 0:4 > (XEN) IRQ64 -> 0:5 > (XEN) IRQ72 -> 0:6 > (XEN) IRQ80 -> 0:7 > (XEN) IRQ88 -> 0:8 > (XEN) IRQ96 -> 0:9 > (XEN) IRQ104 -> 0:10 > (XEN) IRQ112 -> 0:11 > (XEN) IRQ120 -> 0:12 > (XEN) IRQ136 -> 0:13 > (XEN) IRQ144 -> 0:14 > (XEN) IRQ152 -> 0:15 > (XEN) IRQ160 -> 0:16 > (XEN) IRQ168 -> 0:17 > (XEN) IRQ49 -> 0:18 > (XEN) IRQ216 -> 0:20 > (XEN) IRQ33 -> 0:21 > (XEN) IRQ208 -> 0:22 > (XEN) .................................... done.> __ __ _ _ ___ ___ ____ __ _ _____ > \ \/ /___ _ __ | || | / _ \ / _ \ | ___| / _| ___/ |___ / > \ // _ \ ''_ \ | || |_| | | | | | |_|___ \ | |_ / __| | |_ \ > / \ __/ | | | |__ _| |_| | |_| |__|__) || _| (__| |___) | > /_/\_\___|_| |_| |_|(_)___(_)___/ |____(_)_| \___|_|____/ > > (XEN) Xen version 4.0.0 (michael@home) (gcc version 4.4.4 20100630 (Red Hat 4.4.4-10) (GCC) ) Mon Jul 26 23:05:53 BST 2010 > (XEN) Latest ChangeSet: unavailable > (XEN) Command line: nosmp > (XEN) Video information: > (XEN) VGA is text mode 80x25, font 8x16 > (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds > (XEN) Disc information: > (XEN) Found 1 MBR signatures > (XEN) Found 1 EDD information structures > (XEN) Xen-e820 RAM map: > (XEN) 0000000000000000 - 000000000009f000 (usable) > (XEN) 000000000009f000 - 00000000000a0000 (reserved) > (XEN) 0000000000100000 - 00000000df66d800 (usable) > (XEN) 00000000df66d800 - 00000000e0000000 (reserved) > (XEN) 00000000f8000000 - 00000000fc000000 (reserved) > (XEN) 00000000fec00000 - 00000000fec10000 (reserved) > (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) > (XEN) 00000000fed20000 - 00000000fed90000 (reserved) > (XEN) 00000000feda0000 - 00000000feda6000 (reserved) > (XEN) 00000000fee00000 - 00000000fee10000 (reserved) > (XEN) 00000000ffe00000 - 0000000100000000 (reserved) > (XEN) 0000000100000000 - 0000000120000000 (usable) > (XEN) ACPI: RSDP 000FBC60, 0024 (r2 DELL ) > (XEN) ACPI: XSDT DF66F200, 0064 (r1 DELL M08 27D80A10 ASL 61) > (XEN) ACPI: FACP DF66F09C, 00F4 (r4 DELL M08 27D80A10 ASL 61) > (XEN) ACPI: DSDT DF66F800, 5477 (r2 INT430 SYSFexxx 1001 INTL 20050624) > (XEN) ACPI: FACS DF67E000, 0040 > (XEN) ACPI: HPET DF66F300, 0038 (r1 DELL M08 1 ASL 61) > (XEN) ACPI: APIC DF66F400, 0068 (r1 DELL M08 27D80A10 ASL 47) > (XEN) ACPI: MCFG DF66F3C0, 003E (r16 DELL M08 27D80A10 ASL 61) > (XEN) ACPI: SLIC DF66F49C, 0176 (r1 DELL M08 27D80A10 ASL 61) > (XEN) ACPI: OSFR DF66EA00, 002C (r1 DELL M08 27D80A10 ASL 61) > (XEN) ACPI: BOOT DF66EFC0, 0028 (r1 DELL M08 27D80A10 ASL 61) > (XEN) ACPI: SSDT DF66D97E, 04CC (r1 PmRef CpuPm 3000 INTL 20050624) > (XEN) System RAM: 4031MB (4128332kB) > (XEN) Domain heap initialised > (XEN) Processor #0 7:7 APIC version 20 > (XEN) Processor #1 7:7 APIC version 20 > (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 > (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs > (XEN) Using scheduler: SMP Credit Scheduler (credit) > (XEN) Detected 2394.108 MHz processor. > (XEN) Initing memory sharing. > (XEN) VMX: Supported advanced features: > (XEN) - APIC MMIO access virtualisation > (XEN) - APIC TPR shadow > (XEN) - Virtual NMI > (XEN) - MSR direct-access bitmap > (XEN) HVM: ASIDs disabled. > (XEN) HVM: VMX enabled > (XEN) I/O virtualisation disabled > (XEN) ENABLING IO-APIC IRQs > (XEN) -> Using new ACK method > (XEN) Platform timer is 14.318MHz HPET > (XEN) Brought up 1 CPUs > (XEN) *** LOADING DOMAIN 0 *** > (XEN) Xen kernel: 64-bit, lsb, compat32 > (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1a6f000 > (XEN) PHYSICAL MEMORY ARRANGEMENT: > (XEN) Dom0 alloc.: 0000000110000000->0000000118000000 (967255 pages to be allocated) > (XEN) VIRTUAL MEMORY ARRANGEMENT: > (XEN) Loaded kernel: ffffffff81000000->ffffffff81a6f000 > (XEN) Init. ramdisk: ffffffff81a6f000->ffffffff83b37e00 > (XEN) Phys-Mach map: ffffffff83b38000->ffffffff842d92b8 > (XEN) Start info: ffffffff842da000->ffffffff842da4b4 > (XEN) Page tables: ffffffff842db000->ffffffff84300000 > (XEN) Boot stack: ffffffff84300000->ffffffff84301000 > (XEN) TOTAL: ffffffff80000000->ffffffff84400000 > (XEN) ENTRY ADDRESS: ffffffff81819200 > (XEN) Dom0 has maximum 1 VCPUs > (XEN) Scrubbing Free RAM: .done. > (XEN) Xen trace buffers: disabled > (XEN) Std. Loglevel: Errors and warnings > (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings) > (XEN) Xen is relinquishing VGA console. > (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen) > (XEN) Freed 172kB init memory. > (XEN) number of MP IRQ sources: 15. > (XEN) number of IO-APIC #2 registers: 24. > (XEN) testing the IO APIC....................... > (XEN) IO APIC #2...... > (XEN) .... register #00: 00000000 > (XEN) ....... : physical APIC id: 00 > (XEN) ....... : Delivery Type: 0 > (XEN) ....... : LTS : 0 > (XEN) .... register #01: 00170020 > (XEN) ....... : max redirection entries: 0017 > (XEN) ....... : PRQ implemented: 0 > (XEN) ....... : IO APIC version: 0020 > (XEN) .... IRQ redirection table: > (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: > (XEN) 00 000 00 1 0 0 0 0 0 0 00 > (XEN) 01 001 01 0 0 0 0 0 1 1 28 > (XEN) 02 0FF 0F 0 0 0 0 0 1 1 F0 > (XEN) 03 001 01 0 0 0 0 0 1 1 30 > (XEN) 04 001 01 0 0 0 0 0 1 1 38 > (XEN) 05 001 01 0 0 0 0 0 1 1 40 > (XEN) 06 001 01 0 0 0 0 0 1 1 48 > (XEN) 07 001 01 0 0 0 0 0 1 1 50 > (XEN) 08 001 01 0 0 0 0 0 1 1 58 > (XEN) 09 001 01 0 1 0 0 0 1 1 60 > (XEN) 0a 001 01 0 0 0 0 0 1 1 68 > (XEN) 0b 001 01 0 0 0 0 0 1 1 70 > (XEN) 0c 001 01 0 0 0 0 0 1 1 78 > (XEN) 0d 001 01 1 0 0 0 0 1 1 88 > (XEN) 0e 001 01 0 0 0 0 0 1 1 90 > (XEN) 0f 001 01 0 0 0 0 0 1 1 98 > (XEN) 10 001 01 0 1 0 1 0 1 1 A0 > (XEN) 11 001 01 1 1 0 1 0 1 1 A8 > (XEN) 12 001 01 0 1 0 1 0 1 1 31 > (XEN) 13 000 00 1 0 0 0 0 0 0 00 > (XEN) 14 001 01 0 1 0 1 0 1 1 D8 > (XEN) 15 001 01 0 1 0 1 0 1 1 21 > (XEN) 16 001 01 0 1 0 1 0 1 1 D0 > (XEN) 17 000 00 1 0 0 0 0 0 0 00 > (XEN) Using vector-based indexing > (XEN) IRQ to pin mappings: > (XEN) IRQ240 -> 0:2 > (XEN) IRQ40 -> 0:1 > (XEN) IRQ48 -> 0:3 > (XEN) IRQ56 -> 0:4 > (XEN) IRQ64 -> 0:5 > (XEN) IRQ72 -> 0:6 > (XEN) IRQ80 -> 0:7 > (XEN) IRQ88 -> 0:8 > (XEN) IRQ96 -> 0:9 > (XEN) IRQ104 -> 0:10 > (XEN) IRQ112 -> 0:11 > (XEN) IRQ120 -> 0:12 > (XEN) IRQ136 -> 0:13 > (XEN) IRQ144 -> 0:14 > (XEN) IRQ152 -> 0:15 > (XEN) IRQ160 -> 0:16 > (XEN) IRQ168 -> 0:17 > (XEN) IRQ49 -> 0:18 > (XEN) IRQ216 -> 0:20 > (XEN) IRQ33 -> 0:21 > (XEN) IRQ208 -> 0:22 > (XEN) .................................... done.> _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Mon, 23 Aug 2010, Konrad Rzeszutek Wilk wrote:> What amchine is this? You mentioned that this a laptop of some kind. > Looks to be a Dell Inspiron 9400 aka e1705?It is a Dell Inspiron 1525.> that irq 12 and 14 are being both sent to the CPU1 based > on decoding the "Log" and "Phy" flags:. > > We could hack it and make sure that in > setup_IO_APIC_irqs for IRQ 12, we make it send to both CPUs. > Something like this (compile tested only):I don''t get any output from that modification! There were fewer do_IRQ lines but both values still appeared. Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Mon, 23 Aug 2010, M A Young wrote:>> We could hack it and make sure that in >> setup_IO_APIC_irqs for IRQ 12, we make it send to both CPUs. >> Something like this (compile tested only): > > I don''t get any output from that modification!Which might be a 4.0.0 issue not in 4.0.1-rc6 or I might have messed up the test, because I do get the "hack" line segment with 4.0.1-rc6 though the mouse and keyboard still don''t work. Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Aug-24 17:10 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
On Mon, Aug 23, 2010 at 09:37:31PM +0100, M A Young wrote:> On Mon, 23 Aug 2010, M A Young wrote: > > >>We could hack it and make sure that in > >>setup_IO_APIC_irqs for IRQ 12, we make it send to both CPUs. > >>Something like this (compile tested only): > > > >I don''t get any output from that modification! > > Which might be a 4.0.0 issue not in 4.0.1-rc6 or I might have messed > up the test, because I do get the "hack" line segment with 4.0.1-rc6What did the hack line say? Was the value 1, 2, something completly random?> though the mouse and keyboard still don''t work.I spoke with Keir a bit and he mentioned that there was a major rework of the IRQs in 4.0, wherein we would store the IRQs per CPU structure. That is instead of having a "globally" accessible irq structure wherein CPUs might contend for it. The folks at Intel did this so they might have a better idea - and I can dig up from the hg annotate logs who they are and we can ask them. But before we go that route, the other idea was try to disable the irq balance code. Try passing in ''noirqbalance'' and see what happens then. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, 24 Aug 2010, Konrad Rzeszutek Wilk wrote:> What did the hack line say? Was the value 1, 2, something completly > random?(XEN) IO-APIC (apicid-pin) 2-0 hack enabled IRQ 12(78) -> 1. (XEN) , 2-16, 2-17, 2-18, 2-19, 2-20, 2-21, 2-22, 2-23 not connected. Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2010-Aug-24 19:47 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
On 24/08/2010 18:10, "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com> wrote:>> though the mouse and keyboard still don''t work. > > I spoke with Keir a bit and he mentioned that there was a major > rework of the IRQs in 4.0, wherein we would store the IRQs per CPU > structure. That is instead of having a "globally" accessible irq structure > wherein CPUs might contend for it.Specifically the feature is "per-CPU IDTs". This increases the system-wide IRQ limit from ~200 to ~200*nr_cpus. However, there is an increase in complexity in that the interrupt vector assigned to an IRQ is CPU-local, and must be re-allocated when an IRQ''s target is moved between CPUs. The main patch is c/s 20073 by Xiantao Zhang. It is the single biggest IRQ restructuring patch in Xen 4.0 campared with 3.4. -- Keir> The folks at Intel did this so they might > have a better idea - and I can dig up from the hg annotate logs who they > are and we can ask them. But before we go that route, the other idea was > try to disable the irq balance code. Try passing in ''noirqbalance'' and see > what happens_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, 24 Aug 2010, Konrad Rzeszutek Wilk wrote:> But before we go that route, the other idea was try to disable the irq > balance code. Try passing in ''noirqbalance'' and see what happens then.It is very similar. The only significant entries are the do_IRQ lines. The previous run with the hack had (XEN) do_IRQ: 1.120 No irq handler for vector (irq -1) (XEN) do_IRQ: 1.144 No irq handler for vector (irq -1) and this run with the hack and noirqbalance has (XEN) do_IRQ: 1.40 No irq handler for vector (irq -1) (XEN) do_IRQ: 1.120 No irq handler for vector (irq -1) (XEN) do_IRQ: 1.120 No irq handler for vector (irq -1) Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Aug-24 21:16 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
> and this run with the hack and noirqbalance has > (XEN) do_IRQ: 1.40 No irq handler for vector (irq -1)OK, that is vector 0x28, which is IRQ 1: (XEN) 01 001 01 0 0 0 0 0 1 1 28> (XEN) do_IRQ: 1.120 No irq handler for vector (irq -1) > (XEN) do_IRQ: 1.120 No irq handler for vector (irq -1)And 120, is vector 78, which is IRQ 12: (XEN) 0c 001 01 0 0 0 0 0 1 1 78 So both IRQs that are tied to i8042 are deciding that it is time to tell you something. Except that you (well, Xen) isn''t paying any attention. Are they telling you this before the IOAPIC is being printed out? You should use the ''sync_console console_timestamps console_to_ring'' to get the Dom0 and Xen output nicely time synchronized. Or is it done before Dom0 has been started? The c/s 20073 that Keir graciously found does an extra step of programming the IOAPIC and I wonder if doing that is sending the i8042 in some weird state. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, 24 Aug 2010, Konrad Rzeszutek Wilk wrote:> Or is it done before Dom0 has been started? The c/s 20073 that > Keir graciously found does an extra step of programming the IOAPIC and > I wonder if doing that is sending the i8042 in some weird state.The logs are attached. Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Aug-25 14:28 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
On Tue, Aug 24, 2010 at 11:40:37PM +0100, M A Young wrote:> On Tue, 24 Aug 2010, Konrad Rzeszutek Wilk wrote: > > >Or is it done before Dom0 has been started? The c/s 20073 that > >Keir graciously found does an extra step of programming the IOAPIC and > >I wonder if doing that is sending the i8042 in some weird state. > > The logs are attached.Awesome, so> (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: > (XEN) 00 000 00 1 0 0 0 0 0 0 00 > (XEN) 01 001 01 0 0 0 0 0 1 1 28..> (XEN) 0c 001 01 0 0 0 0 0 1 1 78during boot they are set, then as Dom0 boots up it creates its own copies of the IRQs and..> xen: --> irq=2..> xen: --> irq=12 > (XEN) [2010-08-24 23:19:53] IOAPIC[0]: Set PCI routing entry (2-12 -> 0x78 -> IRQ 12 Mode:0 Active:0) > (XEN) [2010-08-24 23:19:53] IOAPIC[0]: Set PCI routing entry (2-1 -> 0x28 -> IRQ 1 Mode:0 Active:0)and the hypercall is done to get the vector number. OK good.> (XEN) [2010-08-24 23:19:53] do_IRQ: 1.40 No irq handler for vector (irq -1) > (XEN) [2010-08-24 23:19:54] do_IRQ: 1.120 No irq handler for vector (irq -1) > (XEN) [2010-08-24 23:19:54] do_IRQ: 1.120 No irq handler for vector (irq -1) > (XEN) [2010-08-24 23:19:54] do_IRQ: 1.120 No irq handler for vector (irq -1)and then we expect the IRQ to work, but the hypervisor has no recollection of us setting those IRQs. So I think we need to ask Bastian for help - he made the changes in the hypervisor so that IRQs below 16 could be programmed by the Dom0, and it might be that we are missing some simple step of setting the per_cpu[vector] field? I am curious if you reverted c/s 21092 whether your machine would work or not. If so, we need to talk to Bastian.> _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wed, 25 Aug 2010, Konrad Rzeszutek Wilk wrote:> So I think we need to ask Bastian for help - he made the changes in the > hypervisor so that IRQs below 16 could be programmed by the Dom0, and it > might be that we are missing some simple step of setting the > per_cpu[vector] field? > > I am curious if you reverted c/s 21092 whether your machine would work > or not. If so, we need to talk to Bastian.Reverting that change "Allow all unused GSI to be configured via IO-APIC by new pv_ops dom0" doesn''t help. The keyboard still doesn''t work. Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Aug-26 14:04 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
On Wed, Aug 25, 2010 at 10:32:47PM +0100, M A Young wrote:> On Wed, 25 Aug 2010, Konrad Rzeszutek Wilk wrote: > > >So I think we need to ask Bastian for help - he made the changes in the > >hypervisor so that IRQs below 16 could be programmed by the Dom0, and it > >might be that we are missing some simple step of setting the > >per_cpu[vector] field? > > > >I am curious if you reverted c/s 21092 whether your machine would work > >or not. If so, we need to talk to Bastian. > > Reverting that change "Allow all unused GSI to be configured via > IO-APIC by new pv_ops dom0" doesn''t help. The keyboard still doesn''t > work.OK, one more thing and if that doens''t work then we (actually, you) will have to get dirty and start debugging this with printks. If you are not too comfortable with this I can help you along. Pull the latest xen-unstable bit, make sure that ''Fix bind_irq_vector() destination'' is in and try that. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2010-Aug-26 14:08 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
On 26/08/2010 15:04, "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com> wrote:>> Reverting that change "Allow all unused GSI to be configured via >> IO-APIC by new pv_ops dom0" doesn''t help. The keyboard still doesn''t >> work. > > OK, one more thing and if that doens''t work then we (actually, you) will > have to get dirty and start debugging this with printks. If you are not > too comfortable with this I can help you along. > > Pull the latest xen-unstable bit, make sure that ''Fix bind_irq_vector() > destination'' is in and try that.No chance. Weirdly that generic sounding function only gets called in one place, for IRQ0 (timer interrupt). -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, 26 Aug 2010, Konrad Rzeszutek Wilk wrote:> OK, one more thing and if that doens''t work then we (actually, you) will > have to get dirty and start debugging this with printks. If you are not > too comfortable with this I can help you along.Okay, here is my first attempt at dirty debugging. I have made a patch to try to track where vector_irq is being changed (attached) and have also attached. I have looked at it quickly, and I don''t think some low IRQs are getting set on the second CPU. Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, 26 Aug 2010, M A Young wrote:> Okay, here is my first attempt at dirty debugging. I have made a patch to try > to track where vector_irq is being changed (attached) and have also attached. > I have looked at it quickly, and I don''t think some low IRQs are getting set > on the second CPU.My next thoughts on this are that almost all IRQs allocated on the first cpu before the second is started aren''t initialized on the second CPU. I presume that __setup_vector_irq from xen/arch/x86/irq.c is where it is supposed to happen void __setup_vector_irq(int cpu) { int irq, vector; struct irq_cfg *cfg; /* Clear vector_irq */ for (vector = 0; vector < NR_VECTORS; ++vector) { per_cpu(vector_irq, cpu)[vector] = -1; printk("__setup_irq_vector: setting vector_irq[%d]=-1 for cpu=%d\n", vector, cpu); } /* Mark the inuse vectors */ for (irq = 0; irq < nr_irqs; ++irq) { cfg = irq_cfg(irq); if (!cpu_isset(cpu, cfg->domain)) continue; vector = irq_to_vector(irq); per_cpu(vector_irq, cpu)[vector] = irq; printk("__setup_irq_vector2: setting vector_irq[%d]=%d for cpu=%d\n", vector, irq, cpu); } } but my debugging suggests it is only actually happening for irq[240] Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, 26 Aug 2010, M A Young wrote:> On Thu, 26 Aug 2010, M A Young wrote: > >> Okay, here is my first attempt at dirty debugging. I have made a patch to >> try to track where vector_irq is being changed (attached) and have also >> attached. I have looked at it quickly, and I don''t think some low IRQs are >> getting set on the second CPU. > > My next thoughts on this are that almost all IRQs allocated on the first cpu > before the second is started aren''t initialized on the second CPU. I presume > that __setup_vector_irq from xen/arch/x86/irq.c is where it is supposed to > happenor perhaps it should happen in io_apic_set_pci_routing from xen/arch/x86/io_apic.c where the higher IRQs are set (it doesn''t because __assign_irq_vector sees the IRQ is already in use old_vector = irq_to_vector(irq); if (old_vector) { cpus_and(tmp_mask, mask, cpu_online_map); cpus_and(tmp_mask, cfg->domain, tmp_mask); if (!cpus_empty(tmp_mask)) { cfg->vector = old_vector; return 0; } } but seems to miss the fact that it is only actually configured for one cpu. Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2010-Aug-27 07:34 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
On 26/08/2010 23:24, "M A Young" <m.a.young@durham.ac.uk> wrote:> or perhaps it should happen in io_apic_set_pci_routing from > xen/arch/x86/io_apic.c where the higher IRQs are set (it doesn''t because > __assign_irq_vector sees the IRQ is already in use > old_vector = irq_to_vector(irq); > if (old_vector) { > cpus_and(tmp_mask, mask, cpu_online_map); > cpus_and(tmp_mask, cfg->domain, tmp_mask); > if (!cpus_empty(tmp_mask)) { > cfg->vector = old_vector; > return 0; > } > } > but seems to miss the fact that it is only actually configured for one > cpu.Setting up for one CPU only was the whole point of the per-cpu IDT patch. It allows the same vector on different CPUs to be bound to different interrupts, hence increasing the system-wide interrupt limit by a factor equal to number of CPUs in the system. As long as the interrupt controller is programmed to only interrupt the CPU which has the vector allocated, all is good. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Aug-31 15:00 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
On Thu, Aug 26, 2010 at 11:24:48PM +0100, M A Young wrote:> On Thu, 26 Aug 2010, M A Young wrote: > > >On Thu, 26 Aug 2010, M A Young wrote: > > > >>Okay, here is my first attempt at dirty debugging. I have made a > >>patch to try to track where vector_irq is being changed > >>(attached) and have also attached. I have looked at it quickly, > >>and I don''t think some low IRQs are getting set on the second > >>CPU.Yeah, that definitly is the problem. Now to why it is happening.> > > >My next thoughts on this are that almost all IRQs allocated on the > >first cpu before the second is started aren''t initialized on the > >second CPU. I presume that __setup_vector_irq from > >xen/arch/x86/irq.c is where it is supposed to happen > or perhaps it should happen in io_apic_set_pci_routing from > xen/arch/x86/io_apic.c where the higher IRQs are set (it doesn''t > because __assign_irq_vector sees the IRQ is already in use > old_vector = irq_to_vector(irq); > if (old_vector) { > cpus_and(tmp_mask, mask, cpu_online_map); > cpus_and(tmp_mask, cfg->domain, tmp_mask); > if (!cpus_empty(tmp_mask)) { > cfg->vector = old_vector; > return 0; > } > } > but seems to miss the fact that it is only actually configured for > one cpu.<nods> That code is actually copied from the Linux kernel .. so that begs the question how does it work under baremetal? Lets recap, on Xen we do this in three stages: 1). Set up all the legacy IRQs. We don''t know yet how many CPUs we have so we just set up the first sixteen IRQs of the IOAPIC to the first CPU (0). We also go through the IDT for CPU0 and set the IDT->IRQ for those legacy IRQs. 2). Then we find out we got more CPUs, and for the other CPUs (1), we setup the IDT and we make the IDT->IRQ(-1). 3). Then when Dom0 starts, we get called for those IRQs once more. And we set the IDT for both CPUs to point to the same IRQ: (XEN) __assign_irq_vector: setting vector_irq[160]=16 for cpu=0 (XEN) __assign_irq_vector: setting vector_irq[160]=16 for cpu=1 except we don''t do it for those that have been already set. So I wonder how this works on baremetal. I''ve an inkling, but looking at the boolean logic it doesn''t make much sense. So this is the file arch/x86/kernel/apic/io_apic.c and the function is: setup_IO_APIC_irq 1442 * For legacy irqs, cfg->domain starts with cpu 0 for legacy 1443 * controllers like 8259. Now that IO-APIC can handle this irq, update 1444 * the cfg->domain. 1445 */ 1446 if (irq < legacy_pic->nr_legacy_irqs && cpumask_test_cpu(0, cfg->domain)) 1447 apic->vector_allocation_domain(0, cfg->domain); 1448 1449 if (assign_irq_vector(irq, cfg, apic->target_cpus())) 1450 return; 1451 1452 dest = apic->cpu_mask_to_apicid_and(cfg->domain, apic->target_cpus()); 1453 1454 apic_printk(APIC_VERBOSE,KERN_DEBUG 1455 "IOAPIC[%d]: Set routing entry (%d-%d -> 0x%x -> " 1456 "IRQ %d Mode:%i Active:%i)\n", 1457 apic_id, mp_ioapics[apic_id].apicid, pin, cfg->vector, 1458 irq, trigger, polarity); If you could, can you instrument it to print the cfg->domain, before the ''vector_allocation_domain'' is called, and as well instrument the assign_irq_vector similary to what you did with Xen? And also instrument the ''dest'' value. Basically the idea is to get an idea of what the per_cpu(vector) gets set during the bootup for legacy IRQs. Similary to what you did with Xen. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, 31 Aug 2010, Konrad Rzeszutek Wilk wrote:> If you could, can you instrument it to print the cfg->domain, before the ''vector_allocation_domain'' > is called, and as well instrument the assign_irq_vector similary to what you did with Xen? > > And also instrument the ''dest'' value. Basically the idea is to get an idea of what the > per_cpu(vector) gets set during the bootup for legacy IRQs. Similary to what you did > with Xen.The kernel code I was working with (2.6.32) doesn''t have the vector_allocation_domain section . I am attaching the debugging output I did get and the patch I used. Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Sep-08 15:44 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
On Fri, Sep 03, 2010 at 07:50:06PM +0100, M A Young wrote:> On Tue, 31 Aug 2010, Konrad Rzeszutek Wilk wrote: > > >If you could, can you instrument it to print the cfg->domain, before the ''vector_allocation_domain'' > >is called, and as well instrument the assign_irq_vector similary to what you did with Xen? > > > >And also instrument the ''dest'' value. Basically the idea is to get an idea of what the > >per_cpu(vector) gets set during the bootup for legacy IRQs. Similary to what you did > >with Xen. > > The kernel code I was working with (2.6.32) doesn''t have the > vector_allocation_domain section . I am attaching the debugging > output I did get and the patch I used.OK, so based on the output the IO APIC pins for you two IRQs should have destination set to 1. .. snip.. _IO_APIC_irq: cfg->domain=-1 setup_IO_APIC_irq: dest=1 IOAPIC[0]: Set routing entry (2-1 -> 0x31 -> IRQ 1 Mode:0 Active:0) setup_IO_APIC_irq: cfg->domain=-1 setup_IO_APIC_irq: dest=1 IOAPIC[0]: Set routing entry (2-12 -> 0x3c -> IRQ 12 Mode:0 Active:0) BUT when the IO APIC is being printed: 01 003 0 0 0 0 0 1 1 31 0c 003 0 0 0 0 0 1 1 3C They are set to dest = 3!? Somehow the IO APIC is being programmed without using the setup_IO_APIC_irq and its friends. Also the per_cpu(vector_irq, 1)[0x31] = 1 is set. So, for the second problem, I think the __setup_vector_irq is the one that sets the vectors on the second CPU to correspond to the right IRQs. But I am not sure how the IOAPIC pin for all IRQs below 16 get set to ''3''. There is something happening between the initial call to init IO_APIC IRQs and when it is being printed that sets the destination to a new value. I''ve piggybacked on your debug patch and added some extra stuff to see if the __setup_vector_irq is responsible for setting the new per_cpu. Those printk''s _might_ not work as all of that is being run on a secondary CPU that is being initialized..? For the IO APIC programming I added a printk/debug_stack by the ioapic_write to see who and when sets those pins on the IOAPIC to 3. Here is the patch: diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c index ec4e874..37482fe 100644 --- a/arch/x86/kernel/apic/io_apic.c +++ b/arch/x86/kernel/apic/io_apic.c @@ -69,7 +69,7 @@ #include <asm/xen/pci.h> #include <asm/xen/pci.h> - +#include <linux/kernel.h> #define __apicdebuginit(type) static type __init #define for_each_irq_pin(entry, head) \ for (entry = head; entry; entry = entry->next) @@ -486,6 +486,11 @@ __ioapic_write_entry(int apic, int pin, struct IO_APIC_route_entry e) void ioapic_write_entry(int apic, int pin, struct IO_APIC_route_entry e) { unsigned long flags; + if (pin == 1 || pin == 0xc) { + printk(KERN_INFO "Reprogramming PIN%d, dest=%d\n", pin, e.dest); + if (e.dest > 1) + dump_stack(); + } spin_lock_irqsave(&ioapic_lock, flags); __ioapic_write_entry(apic, pin, e); spin_unlock_irqrestore(&ioapic_lock, flags); @@ -1198,6 +1203,7 @@ __assign_irq_vector(int irq, struct irq_cfg *cfg, const struct cpumask *mask) if (old_vector) { cpumask_and(tmp_mask, mask, cpu_online_mask); cpumask_and(tmp_mask, cfg->domain, tmp_mask); + printk(KERN_INFO "old_vector: %d mask: %x\n", old_vector, tmp_mask->bits[0]); if (!cpumask_empty(tmp_mask)) { free_cpumask_var(tmp_mask); return 0; @@ -1214,6 +1220,8 @@ __assign_irq_vector(int irq, struct irq_cfg *cfg, const struct cpumask *mask) vector = current_vector; offset = current_offset; + printk(KERN_INFO "vector: %d, mask: %x, cpu: %d per_cpu:%x\n", + vector, tmp_mask->bits[0], cpu, per_cpu(vector_irq, cpu)[vector]); next: vector += 8; if (vector >= first_system_vector) { @@ -1237,8 +1245,11 @@ next: cfg->move_in_progress = 1; cpumask_copy(cfg->old_domain, cfg->domain); } - for_each_cpu_and(new_cpu, tmp_mask, cpu_online_mask) + for_each_cpu_and(new_cpu, tmp_mask, cpu_online_mask) { per_cpu(vector_irq, new_cpu)[vector] = irq; + printk(KERN_WARNING "__assign_irq_vector: setting vector_irq[%d]=%d for cpu=%d\n", + vector, irq, new_cpu); + } cfg->vector = vector; cpumask_copy(cfg->domain, tmp_mask); err = 0; @@ -1304,6 +1315,8 @@ void __setup_vector_irq(int cpu) if (!cpumask_test_cpu(cpu, cfg->domain)) continue; vector = cfg->vector; + printk(KERN_INFO "%s: vector: %d on CPU %d set to IRQ: %d\n", + __FUNCTION__, vector, cpu, irq); per_cpu(vector_irq, cpu)[vector] = irq; } /* Mark the free vectors */ @@ -1313,8 +1326,11 @@ void __setup_vector_irq(int cpu) continue; cfg = irq_cfg(irq); - if (!cpumask_test_cpu(cpu, cfg->domain)) + if (!cpumask_test_cpu(cpu, cfg->domain)) { + printk(KERN_INFO "%s: vector %d on CPU %d reset b/c not in affinity mask (%d)\n", + __FUNCTION__, vector, cpu, cfg->domain->bits[0]); per_cpu(vector_irq, cpu)[vector] = -1; + } } } @@ -1452,7 +1468,20 @@ int setup_ioapic_entry(int apic_id, int irq, entry->mask = 1; return 0; } - +static void dump_vectors(const char *prefix) { + int cpu; + int vector; + + for (vector = 0x30; vector < 0x3f; vector++) { + for_each_cpu_and(cpu, 0xff, cpu_online_mask) { + if (per_cpu(vector_irq, cpu)[vector] != -1) + printk(KERN_INFO "%s [vec:%d,cpu:%d] = irq:%d\n", + prefix, + vector, cpu, + per_cpu(vector_irq, cpu)[vector]); + } + } +} static void setup_IO_APIC_irq(int apic_id, int pin, unsigned int irq, struct irq_desc *desc, int trigger, int polarity) { @@ -1465,10 +1494,15 @@ static void setup_IO_APIC_irq(int apic_id, int pin, unsigned int irq, struct irq cfg = desc->chip_data; + printk(KERN_WARNING "setup_IO_APIC_irq: cfg->domain=%d (vector: %d)\n", cfg->domain->bits[0], cfg->vector); + + dump_vectors("PRE"); if (assign_irq_vector(irq, cfg, apic->target_cpus())) return; + dump_vectors("PAST"); dest = apic->cpu_mask_to_apicid_and(cfg->domain, apic->target_cpus()); + printk(KERN_WARNING "setup_IO_APIC_irq: dest=%d\n", dest); apic_printk(APIC_VERBOSE,KERN_DEBUG "IOAPIC[%d]: Set routing entry (%d-%d -> 0x%x -> " _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wed, 8 Sep 2010, Konrad Rzeszutek Wilk wrote:> I''ve piggybacked on your debug patch and added some extra stuff to see if the __setup_vector_irq > is responsible for setting the new per_cpu. Those printk''s _might_ not work as > all of that is being run on a secondary CPU that is being initialized..? > > For the IO APIC programming I added a printk/debug_stack by the ioapic_write > to see who and when sets those pins on the IOAPIC to 3.I had to modify the patch slightly to get the kernel not to panic, but it did mostly work after that. The results are attached. Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Sep-08 23:17 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
On Wed, Sep 08, 2010 at 10:36:30PM +0100, M A Young wrote:> On Wed, 8 Sep 2010, Konrad Rzeszutek Wilk wrote: > > >I''ve piggybacked on your debug patch and added some extra stuff to see if the __setup_vector_irq > >is responsible for setting the new per_cpu. Those printk''s _might_ not work as > >all of that is being run on a secondary CPU that is being initialized..? > > > >For the IO APIC programming I added a printk/debug_stack by the ioapic_write > >to see who and when sets those pins on the IOAPIC to 3. > > I had to modify the patch slightly to get the kernel not to panic, > but it did mostly work after that. The results are attached.Hm, and they still don''t give a hint of who sets the IOAPIC pin 1-through 15 to destination 3.> setup_IO_APIC_irq: dest=1 > IOAPIC[0]: Set routing entry (2-1 -> 0x31 -> IRQ 1 Mode:0 Active:0) > Reprogramming PIN1, dest=1 > setup_IO_APIC_irq: cfg->domain=-1 (vector: 48)Excellent, IOAPIC PIN1 should have dest 1... but> NR Dst Mask Trig IRR Pol Stat Dmod Deli Vect: > 00 000 1 0 0 0 0 0 0 00 > 01 003 0 0 0 0 0 1 1 31It is 3 here? And then later we set the destination to 3. ..> setup_IO_APIC_irq: dest=3 > IOAPIC[0]: Set routing entry (2-12 -> 0x3c -> IRQ 12 Mode:0 Active:0) > Reprogramming PIN12, dest=3 > Pid: 1, comm: swapper Not tainted 2.6.32.21 #1 > Call Trace: > [<ffffffff8102b25d>] ioapic_write_entry+0x48/0xda > [<ffffffff8102bdd9>] setup_IO_APIC_irq+0x21f/0x22e > [<ffffffff8144680a>] ? irq_to_desc_alloc_node+0x16/0x188 > [<ffffffff8102bf39>] io_apic_set_pci_routing+0x151/0x17c > [<ffffffff810263ae>] mp_register_gsi+0x17d/0x18f > [<ffffffff8102644d>] acpi_register_gsi+0x5b/0x64 > [<ffffffff812a02f3>] pnpacpi_parse_allocated_irqresource+0x113/0x14aCompared this to how Xen works it is pretty much exactly the same. Except that I think Xen forces the IO APIC to be reset. Here it looks as if it is untouched from the boot - but who knows? I think you need to figure out who sets the IO APIC entries earlier on. It might be as well that nobody does and this is what the BIOS came up and under Xen we clear it up. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wed, 8 Sep 2010, Konrad Rzeszutek Wilk wrote:> Compared this to how Xen works it is pretty much exactly the same. > Except that I think Xen forces the IO APIC to be reset. Here it looks > as if it is untouched from the boot - but who knows? > > I think you need to figure out who sets the IO APIC entries earlier > on. It might be as well that nobody does and this is what the BIOS > came up and under Xen we clear it up.The latest 4.0.2-rc1-pre fixes my problem. I saw the patch http://xenbits.xensource.com/xen-4.0-testing.hg?rev/965d47d5d7c2 which looked relevant, so I tried a build including it and my laptop keyboard and mouse work again. Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Sep-20 14:51 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse
On Fri, Sep 17, 2010 at 11:49:02PM +0100, M A Young wrote:> On Wed, 8 Sep 2010, Konrad Rzeszutek Wilk wrote: > > >Compared this to how Xen works it is pretty much exactly the same. > >Except that I think Xen forces the IO APIC to be reset. Here it looks > >as if it is untouched from the boot - but who knows? > > > >I think you need to figure out who sets the IO APIC entries earlier > >on. It might be as well that nobody does and this is what the BIOS > >came up and under Xen we clear it up. > > The latest 4.0.2-rc1-pre fixes my problem. I saw the patch > http://xenbits.xensource.com/xen-4.0-testing.hg?rev/965d47d5d7c2 > which looked relevant, so I tried a build including it and my laptop > keyboard and mouse work again.Woot! Excellent! _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Pasi Kärkkäinen
2010-Sep-20 15:05 UTC
Re: [Xen-devel] xen 4 only seeing one keyboard and mouse, fixed in xen 4.0.2-rc-pre
On Mon, Sep 20, 2010 at 10:51:52AM -0400, Konrad Rzeszutek Wilk wrote:> On Fri, Sep 17, 2010 at 11:49:02PM +0100, M A Young wrote: > > On Wed, 8 Sep 2010, Konrad Rzeszutek Wilk wrote: > > > > >Compared this to how Xen works it is pretty much exactly the same. > > >Except that I think Xen forces the IO APIC to be reset. Here it looks > > >as if it is untouched from the boot - but who knows? > > > > > >I think you need to figure out who sets the IO APIC entries earlier > > >on. It might be as well that nobody does and this is what the BIOS > > >came up and under Xen we clear it up. > > > > The latest 4.0.2-rc1-pre fixes my problem. I saw the patch > > http://xenbits.xensource.com/xen-4.0-testing.hg?rev/965d47d5d7c2 > > which looked relevant, so I tried a build including it and my laptop > > keyboard and mouse work again. > > Woot! Excellent! >Yeah, good to hear this got resolved. Debian folks might want to include that patch in their Xen .deb packages for upcoming Squeeze (6.0) release.. -- Pasi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel