Hi,
I''m facing some problem when I''m trying to work with serial
console with
Xen. I followed the instructions from user manual
(http:www.cl.cam.ac.uk/Research/SRG/netos/xen/readmes/user/user.html) and
constructed the grub.conf accordingly. I''m attaching the grub.conf file
here and would be referring to it in the next few lines. The problems faced
are as follows:
1. I can boot normal Linux FC4 kernel and direct its output to serial
console (and see the messages in a separate monitor) using the options as
mentioned in the grub.conf.
If I do the exactly the same (meaning, whatever you can see as
uncommeted lines in the xen-related portion of the grub.conf),
I can''t see the messages in the separate monitor.
2. If I uncomment the third line in the following:
title Fedora Core (Mar 17-xen0)
root (hd0,0)
# kernel /xen.gz dom0_mem=131072 com1=9600,8n1
kernel /xen.gz dom0_mem=131072
module /vmlinuz-2.6.12.6-xen0 ro root=LABEL=/ rhgb quiet
console=tty0 console=ttyS0,9600n8
module /initrd-2.6.12.6-xen0.img
then Xen even does not boot up. I''m not sure if I put the correct entry
in
the third line, though.
Can anyone kindly point out my mistake or tell me if I''m missing
something?
I wanted to see the messages in the different monitor (conmnected via serial
console) for the purpose of debugging. Hence, it''d be nice if someone
can
let me know easier way of debugging Xen kernel code.
Thanks in advance,
Amitayu
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Stephen C. Tweedie
2006-Apr-07 13:27 UTC
Re: [Xen-devel] Serial console problem with Xen 3.0
Hi, On Wed, 2006-04-05 at 00:40 -0400, Amitayu Das wrote:> 1. I can boot normal Linux FC4 kernel and direct its output to serial > console (and see the messages in a separate monitor) using the options > as mentioned in the grub.conf. > If I do the exactly the same (meaning, whatever you can see as > uncommeted lines in the xen-related portion of the grub.conf), > I can''t see the messages in the separate monitor.Right, because you commented out the xen.gz com1=... line, so there''s no HV serial console. You need that enabled.> 2. If I uncomment the third line in the following: > # kernel /xen.gz dom0_mem=131072 com1=9600,8n1> then Xen even does not boot up.I''ve seen this a lot --- for me, adding "sync_console" to that xen.gz line cures it. Does that help for you? Thanks, Stephen _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stephen C. Tweedie wrote:> On Wed, 2006-04-05 at 00:40 -0400, Amitayu Das wrote: > >> 1. I can boot normal Linux FC4 kernel and direct its output to serial >> console (and see the messages in a separate monitor) using the options >> as mentioned in the grub.conf. >> If I do the exactly the same (meaning, whatever you can see as >> uncommeted lines in the xen-related portion of the grub.conf), >> I can''t see the messages in the separate monitor. > > Right, because you commented out the xen.gz com1=... line, so there''s no > HV serial console. You need that enabled. > >> 2. If I uncomment the third line in the following: >> # kernel /xen.gz dom0_mem=131072 com1=9600,8n1 > >> then Xen even does not boot up. > > I''ve seen this a lot --- for me, adding "sync_console" to that xen.gz > line cures it. Does that help for you?Me too. Adding "sync_console" at least repaired the console *output*. I still have the problem that keyboard input does not reach linux, as it seems. Here is my config: serial --unit=0 --speed=38400 --word=8 --parity=no --stop=1 terminal --timeout=10 serial console title CentOS (2.6.16-2.EL4xen0) root (hd0,0) kernel /xen.gz-2.6.16-2.EL4 com1=38400,8n1 conswitch=s sync_console module /vmlinuz-2.6.16-2.EL4xen0 ro root=LABEL=/1 console=tty0 console=ttyS0,38400n8 module /initrd-2.6.16-2.EL4xen0.img Using the same config for native linux gives me a working console access (NB. I have agetty in inittab). In Xen/Linux I get all boot messages (from linux and Xen), but when I hit the login prompt, no keyboard input is excepted. I am using minicom on the other side of the cable. Additionally, pressing CTRL-A (or CTRL-S after changing the key) three times does *not* give me access to the Xen console. Any additional hints how I could get this working? (NB. this is xen-unstable, just a two days old, on x86 with an Intel 7210 board). Best Regards, Michael Paesold _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > Me too. Adding "sync_console" at least repaired the console *output*. I > still have the problem that keyboard input does not reach linux, as it > seems. Here is my config: >Adding "pnpacpi=off" to the command line is our quick and dirty hack for now. Don''t know the root cause yet, but at least it will get you going. Cheers, Don _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi,
I finally got it working. Let me mention the steps below. Please point out
if I''m doing something wrong or weird:
1. We need to config the kernel to support this functionality and recompile
krenel for this. In order to do that, follow these steps:
a. go to xen root directory
b. go to linux-2.6.12-xen0
c. make ARCH=xen menuconfig
d. enable options for
Device Driver --> Character Device -->Serial Drivers
Enable appropriate options and save the new configrations.
e. recompile the kernel and install the new kernel.
f. Put the following entries into the /etc/grub.conf:
serial --unit=0 --speed=9600 --word=8 --parity=no --stop=1
terminal --timeout=10 serial console
title Ourxen (Mar 17-xen0)
root (hd0,0)
kernel /xen.gz dom0_mem=131072
module /vmlinuz-2.6.12.6-xen0 ro root=LABEL=/ rhgb quiet
console=tty0 console=ttyS0,9600n8
module /initrd-2.6.12.6-xen0.img
g. reboot the machine with the new image
Currently, this is working for me. If someone can tell me a better way to do
the things, that''d be great !!
Amitayu
On 4/7/06, Michael Paesold <mpaesold@gmx.at>
wrote:>
> Stephen C. Tweedie wrote:
> > On Wed, 2006-04-05 at 00:40 -0400, Amitayu Das wrote:
> >
> >> 1. I can boot normal Linux FC4 kernel and direct its output to
serial
> >> console (and see the messages in a separate monitor) using the
options
> >> as mentioned in the grub.conf.
> >> If I do the exactly the same (meaning, whatever you can see as
> >> uncommeted lines in the xen-related portion of the grub.conf),
> >> I can''t see the messages in the separate monitor.
> >
> > Right, because you commented out the xen.gz com1=... line, so
there''s no
> > HV serial console. You need that enabled.
> >
> >> 2. If I uncomment the third line in the following:
> >> # kernel /xen.gz dom0_mem=131072 com1=9600,8n1
> >
> >> then Xen even does not boot up.
> >
> > I''ve seen this a lot --- for me, adding
"sync_console" to that xen.gz
> > line cures it. Does that help for you?
>
> Me too. Adding "sync_console" at least repaired the console
*output*. I
> still have the problem that keyboard input does not reach linux, as it
> seems. Here is my config:
>
> serial --unit=0 --speed=38400 --word=8 --parity=no --stop=1
> terminal --timeout=10 serial console
> title CentOS (2.6.16-2.EL4xen0)
> root (hd0,0)
> kernel /xen.gz-2.6.16-2.EL4 com1=38400,8n1 conswitch=s
> sync_console
> module /vmlinuz-2.6.16-2.EL4xen0 ro root=LABEL=/1 console=tty0
> console=ttyS0,38400n8
> module /initrd-2.6.16-2.EL4xen0.img
>
> Using the same config for native linux gives me a working console access
> (NB. I have agetty in inittab). In Xen/Linux I get all boot messages (from
> linux and Xen), but when I hit the login prompt, no keyboard input is
> excepted. I am using minicom on the other side of the cable.
>
> Additionally, pressing CTRL-A (or CTRL-S after changing the key) three
> times
> does *not* give me access to the Xen console.
>
> Any additional hints how I could get this working? (NB. this is
> xen-unstable, just a two days old, on x86 with an Intel 7210 board).
>
> Best Regards,
> Michael Paesold
>
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
On 7 Apr 2006, at 15:44, Don Zickus wrote:>> Me too. Adding "sync_console" at least repaired the console *output*. >> I >> still have the problem that keyboard input does not reach linux, as it >> seems. Here is my config: >> > > Adding "pnpacpi=off" to the command line is our quick and dirty hack > for > now. Don''t know the root cause yet, but at least it will get you > going.I have some suspicion that this, and the IRQ flooding issues that suse have been seeing, may be due to insufficient checking and translation and book-keeping at the ioapic management interface that Xen exports. See ioapic_guest_write() in xen/arch/x86/io_apic.c. Domain0 is responsible for IRQ steering and routing, and therefore for most of the management of IO-APICs, *but*: 1. Xen is responsible for masking/unmasking as part of normal interrupt servicing. 2. Xen may own some of the interrupts routed via the IO-APIC (serial and timer). Domain0 won''t necessarily know about these and I think in the case of serial interrupt in particular it may end up screwing that IRQ line, in theory. It may be worth getting Xen and XenLinux to dump verbose info about their view of IO-APIC state, and maybe add tracing to ioapic_guest_write() and see how domain0 is trying to drive the IO APICs. You may well see some IO APIC write coming thru just before the serial line goes dead. If so, the question will be whether the problem is due to broken checking/translating in Xen, or whether a fix is needed to dom0 ioapic code. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser wrote:> On 7 Apr 2006, at 15:44, Don Zickus wrote: > >>> Me too. Adding "sync_console" at least repaired the console *output*. I >>> still have the problem that keyboard input does not reach linux, as it >>> seems. Here is my config: >>> >> >> Adding "pnpacpi=off" to the command line is our quick and dirty hack >> >> for >> now. Don''t know the root cause yet, but at least it will get you going. > > I have some suspicion that this, and the IRQ flooding issues that > suse have been seeing, may be due to insufficient checking and > translation and book-keeping at the ioapic management interface that Xen > exports.... I just wanted to confirm that pnpacpi=off solves my problem and give me full access over the serial console. Should this get an issue in the bugtracker? Best Regards, Michael Paesold _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 7 Apr 2006, at 17:17, Michael Paesold wrote:>> I have some suspicion that this, and the IRQ flooding issues that >> suse have been seeing, may be due to insufficient checking and >> translation and book-keeping at the ioapic management interface that >> Xen exports. > ... > > I just wanted to confirm that pnpacpi=off solves my problem and give > me full access over the serial console. > > Should this get an issue in the bugtracker?Yes, it''s probably a good thing to gather all this stuff in one place. By the way, I have some fixes already for the IO-APIC checking routine that I mentioned. We''re just going to give it a shakedown in Cambridge before putting it in the public repository. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stephen C. Tweedie
2006-Apr-07 17:54 UTC
Re: [Xen-devel] Serial console problem with Xen 3.0
Hi, On Fri, 2006-04-07 at 15:59 +0200, Michael Paesold wrote:> Me too. Adding "sync_console" at least repaired the console *output*. I > still have the problem that keyboard input does not reach linuxSerial input seems to have a bad interaction with ACPI plug-n-play. Appending "pnpacpi=off" to the "module vmlinuz..." grub.conf line should turn that off, and restore serial input. --Stephen _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 7 Apr 2006, at 17:51, Keir Fraser wrote:>> I just wanted to confirm that pnpacpi=off solves my problem and give >> me full access over the serial console. >> >> Should this get an issue in the bugtracker? > > Yes, it''s probably a good thing to gather all this stuff in one place. > By the way, I have some fixes already for the IO-APIC checking routine > that I mentioned. We''re just going to give it a shakedown in Cambridge > before putting it in the public repository.I can confirm that my patch at least fixes the pnpacpi problem that you''ve been seeing. I''ve also enabled CONFIG_PNP_ACPI by default in our -xen0 and -xen defconfig files, so we should get wider feedback on issues relating to that config option now. I''ve also backported my io_apic.c changes to the 3.0-testing tree so they should be in 3.0.2-1 (the next rolling release of 3.0.2). -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser wrote:> > On 7 Apr 2006, at 17:51, Keir Fraser wrote: > >>> I just wanted to confirm that pnpacpi=off solves my problem and give >>> me full access over the serial console. >>> >>> Should this get an issue in the bugtracker? >> >> Yes, it''s probably a good thing to gather all this stuff in one place. >> By the way, I have some fixes already for the IO-APIC checking routine >> that I mentioned. We''re just going to give it a shakedown in Cambridge >> before putting it in the public repository. > > I can confirm that my patch at least fixes the pnpacpi problem that > you''ve been seeing. I''ve also enabled CONFIG_PNP_ACPI by default in our > -xen0 and -xen defconfig files, so we should get wider feedback on > issues relating to that config option now.Great, I am going to pull from xen-unstable after the weekend and test the changes. Thanks to Don and Stephen for the "pnpacpi=off" hint. Best Regards, Michael Paesold _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Fri, 2006-04-07 at 10:44 -0400, Don Zickus wrote:> > > > Me too. Adding "sync_console" at least repaired the console *output*. I > > still have the problem that keyboard input does not reach linux, as it > > seems. Here is my config: > > > > Adding "pnpacpi=off" to the command line is our quick and dirty hack for > now. Don''t know the root cause yet, but at least it will get you going. > > Cheers, > DonI had a similar problem during some of my development. Whenever I booted Xen with a serial console, the serial output would quit shortly after dom0 booted and then the system would "hang" at random places. I tracked it down to the pnpacpi code as well and I think I know (now) what the problem is. ACPI provides a means to disable the devices (the _DIS method). Serial ports often appear as ACPI devices and when pnpacpi discovers that no driver is using a device, it disables it (by calling the _DIS method). Xen thinks it has exclusive access to the serial port (I believe it blocks dom0 from accessing the related i/o ports), but dom0 has an out-of-band way (through ACPI) of messing with the serial port (_DIS isn''t the only method in ACPI that can be used to change the properties of the serial port). I believe my system was subsequently hanging because the buffer to the serial port got full and it was waiting for it to empty (although I didn''t ever verify this part). I''ve not yet found a good workaround except to disable pnpacpi. It would be nice to have pnpacpi (or some component) detect that Xen is using the serial port and leave it untouched. I think this explains the behavior in bug 229 (and possibly 579 as well). Ryan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 11 Apr 2006, at 16:38, Ryan wrote:> I''ve not yet found a good workaround except to disable pnpacpi. It > would > be nice to have pnpacpi (or some component) detect that Xen is using > the > serial port and leave it untouched. > > I think this explains the behavior in bug 229 (and possibly 579 as > well).It''s actually fixed in latest -unstable and -testing trees (we simply ignore the IO-APIC write that attempts to mask the interrupt). -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, 2006-04-11 at 16:42 +0100, Keir Fraser wrote:> On 11 Apr 2006, at 16:38, Ryan wrote: > > > I''ve not yet found a good workaround except to disable pnpacpi. It > > would > > be nice to have pnpacpi (or some component) detect that Xen is using > > the > > serial port and leave it untouched. > > > > I think this explains the behavior in bug 229 (and possibly 579 as > > well). > > It''s actually fixed in latest -unstable and -testing trees (we simply > ignore the IO-APIC write that attempts to mask the interrupt). > > -- Keir >Hmm... I hadn''t tried the latest xen-unstable with that changeset and you''re right, it appears to work now. Is that how the ACPI tables specify how to disable the serial port? Or is my conclusion about ACPI completely unrelated to why it wasn''t working? Ryan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 11 Apr 2006, at 17:00, Ryan wrote:> Hmm... I hadn''t tried the latest xen-unstable with that changeset and > you''re right, it appears to work now. Is that how the ACPI tables > specify how to disable the serial port? Or is my conclusion about ACPI > completely unrelated to why it wasn''t working?Your analysis was mostly correct I believe. However I don''t think pnpacpi is even explicitly disabling the device or interrupt line. It''s calling into IO-APIC code to set up the routing entry and that code is deciding to mask the entry because it knows that no device driver is currently registered on that irq. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel