Xen devel - Sep 2012 - [PATCH] xen/console: introduce a 'w' debug-key that dumps the console ring

This patch adds a new ''w'' debug-key, chosen from the limited
remaining
keys only due to its proximity to ''q'', that dumps the console
ring to
configured console devices. It''s useful to for tracking down how an
unresponsive system got into a broken state via serial console.

Signed-off-by: Matt Wilson <msw@amazon.com>

diff -r bd953fda6106 -r 20f6976e28a1 xen/drivers/char/console.c
--- a/xen/drivers/char/console.c	Fri Sep 28 10:59:41 2012 +0200
+++ b/xen/drivers/char/console.c	Sat Sep 29 05:00:05 2012 +0000
@@ -264,6 +264,49 @@ static void sercon_puts(const char *s)
         serial_puts(sercon_handle, s);
 }
 
+static void dump_console_ring_key(unsigned char key)
+{
+    uint32_t idx, len, sofar, c;
+    unsigned int order;
+    char *buf;
+
+    printk("''%c'' pressed -> dumping console ring
buffer (dmesg)\n", key);
+
+    /* create a buffer in which we''ll copy the ring in the correct
+       order and NUL terminate */
+    order = get_order_from_bytes(conring_size + 1);
+    buf = alloc_xenheap_pages(order, 0);
+    if ( buf == NULL )
+    {
+        printk("unable to allocate memory!\n");
+        return;
+    }
+
+    c = conringc;
+    sofar = 0;
+    while ( (c != conringp) )
+    {
+        idx = CONRING_IDX_MASK(c);
+        len = conringp - c;
+        if ( (idx + len) > conring_size )
+            len = conring_size - idx;
+        memcpy(buf + sofar, &conring[idx], len);
+        sofar += len;
+        c += len;
+    }
+    buf[sofar] = ''\0'';
+
+    sercon_puts(buf);
+    vga_puts(buf);
+
+    free_xenheap_pages(buf, order);
+}
+
+static struct keyhandler dump_console_ring_keyhandler = {
+    .u.fn = dump_console_ring_key,
+    .desc = "synchronously dump console ring buffer (dmesg)"
+};
+
 /* CTRL-<switch_char> switches input direction between Xen and DOM0. */
 #define switch_code (opt_conswitch[0]-''a''+1)
 static int __read_mostly xen_rx = 1; /* FALSE => serial input passed to
domain 0. */
@@ -661,6 +704,8 @@ void __init console_endboot(void)
     if ( opt_conswitch[1] == ''x'' )
         xen_rx = !xen_rx;
 
+    register_keyhandler(''w'',
&dump_console_ring_keyhandler);
+
     /* Serial input is directed to DOM0 by default. */
     switch_serial_input();
 }

On 29/09/2012 06:02, "Matt Wilson" <msw@amazon.com> wrote:
> This patch adds a new ''w'' debug-key, chosen from the
limited remaining
> keys only due to its proximity to ''q'', that dumps the
console ring to
> configured console devices. It''s useful to for tracking down how
an
> unresponsive system got into a broken state via serial console.
Why wouldn''t everything on the ring already have been sent to the
configured
console devices?

 -- Keir
> Signed-off-by: Matt Wilson <msw@amazon.com>
> 
> diff -r bd953fda6106 -r 20f6976e28a1 xen/drivers/char/console.c
> --- a/xen/drivers/char/console.c Fri Sep 28 10:59:41 2012 +0200
> +++ b/xen/drivers/char/console.c Sat Sep 29 05:00:05 2012 +0000
> @@ -264,6 +264,49 @@ static void sercon_puts(const char *s)
>          serial_puts(sercon_handle, s);
>  }
>  
> +static void dump_console_ring_key(unsigned char key)
> +{
> +    uint32_t idx, len, sofar, c;
> +    unsigned int order;
> +    char *buf;
> +
> +    printk("''%c'' pressed -> dumping console ring
buffer (dmesg)\n", key);
> +
> +    /* create a buffer in which we''ll copy the ring in the
correct
> +       order and NUL terminate */
> +    order = get_order_from_bytes(conring_size + 1);
> +    buf = alloc_xenheap_pages(order, 0);
> +    if ( buf == NULL )
> +    {
> +        printk("unable to allocate memory!\n");
> +        return;
> +    }
> +
> +    c = conringc;
> +    sofar = 0;
> +    while ( (c != conringp) )
> +    {
> +        idx = CONRING_IDX_MASK(c);
> +        len = conringp - c;
> +        if ( (idx + len) > conring_size )
> +            len = conring_size - idx;
> +        memcpy(buf + sofar, &conring[idx], len);
> +        sofar += len;
> +        c += len;
> +    }
> +    buf[sofar] = ''\0'';
> +
> +    sercon_puts(buf);
> +    vga_puts(buf);
> +
> +    free_xenheap_pages(buf, order);
> +}
> +
> +static struct keyhandler dump_console_ring_keyhandler = {
> +    .u.fn = dump_console_ring_key,
> +    .desc = "synchronously dump console ring buffer (dmesg)"
> +};
> +
>  /* CTRL-<switch_char> switches input direction between Xen and DOM0.
*/
>  #define switch_code (opt_conswitch[0]-''a''+1)
>  static int __read_mostly xen_rx = 1; /* FALSE => serial input passed to
> domain 0. */
> @@ -661,6 +704,8 @@ void __init console_endboot(void)
>      if ( opt_conswitch[1] == ''x'' )
>          xen_rx = !xen_rx;
>  
> +    register_keyhandler(''w'',
&dump_console_ring_keyhandler);
> +
>      /* Serial input is directed to DOM0 by default. */
>      switch_serial_input();
>  }

On Sat, Sep 29, 2012 at 06:55:41AM +0100, Keir Fraser
wrote:
> On 29/09/2012 06:02, "Matt Wilson" <msw@amazon.com> wrote:
> 
> > This patch adds a new ''w'' debug-key, chosen from the
limited remaining
> > keys only due to its proximity to ''q'', that dumps
the console ring to
> > configured console devices. It''s useful to for tracking down
how an
> > unresponsive system got into a broken state via serial console.
> 
> Why wouldn''t everything on the ring already have been sent to the
configured
> console devices?
It would have been. But what if no one was looking, or dom0 cleared
the console, or placed it into a scrolling mode that prevents a
virtual terminal from placing output into a scrollback buffer. With
this patch, you can connect to a serial console after a machine
becomes unresponsive and gather clues on what went wrong.

Matt
> > Signed-off-by: Matt Wilson <msw@amazon.com>
> > 
> > diff -r bd953fda6106 -r 20f6976e28a1 xen/drivers/char/console.c
> > --- a/xen/drivers/char/console.c Fri Sep 28 10:59:41 2012 +0200
> > +++ b/xen/drivers/char/console.c Sat Sep 29 05:00:05 2012 +0000
> > @@ -264,6 +264,49 @@ static void sercon_puts(const char *s)
> >          serial_puts(sercon_handle, s);
> >  }
> >  
> > +static void dump_console_ring_key(unsigned char key)
> > +{
> > +    uint32_t idx, len, sofar, c;
> > +    unsigned int order;
> > +    char *buf;
> > +
> > +    printk("''%c'' pressed -> dumping console
ring buffer (dmesg)\n", key);
> > +
> > +    /* create a buffer in which we''ll copy the ring in the
correct
> > +       order and NUL terminate */
> > +    order = get_order_from_bytes(conring_size + 1);
> > +    buf = alloc_xenheap_pages(order, 0);
> > +    if ( buf == NULL )
> > +    {
> > +        printk("unable to allocate memory!\n");
> > +        return;
> > +    }
> > +
> > +    c = conringc;
> > +    sofar = 0;
> > +    while ( (c != conringp) )
> > +    {
> > +        idx = CONRING_IDX_MASK(c);
> > +        len = conringp - c;
> > +        if ( (idx + len) > conring_size )
> > +            len = conring_size - idx;
> > +        memcpy(buf + sofar, &conring[idx], len);
> > +        sofar += len;
> > +        c += len;
> > +    }
> > +    buf[sofar] = ''\0'';
> > +
> > +    sercon_puts(buf);
> > +    vga_puts(buf);
> > +
> > +    free_xenheap_pages(buf, order);
> > +}
> > +
> > +static struct keyhandler dump_console_ring_keyhandler = {
> > +    .u.fn = dump_console_ring_key,
> > +    .desc = "synchronously dump console ring buffer
(dmesg)"
> > +};
> > +
> >  /* CTRL-<switch_char> switches input direction between Xen and
DOM0. */
> >  #define switch_code (opt_conswitch[0]-''a''+1)
> >  static int __read_mostly xen_rx = 1; /* FALSE => serial input
passed to
> > domain 0. */
> > @@ -661,6 +704,8 @@ void __init console_endboot(void)
> >      if ( opt_conswitch[1] == ''x'' )
> >          xen_rx = !xen_rx;
> >  
> > +    register_keyhandler(''w'',
&dump_console_ring_keyhandler);
> > +
> >      /* Serial input is directed to DOM0 by default. */
> >      switch_serial_input();
> >  }
>

>>> On 29.09.12 at 20:49, Matt Wilson <msw@amazon.com> wrote:
> On Sat, Sep 29, 2012 at 06:55:41AM +0100, Keir Fraser wrote:
>> On 29/09/2012 06:02, "Matt Wilson" <msw@amazon.com>
wrote:
>> 
>> > This patch adds a new ''w'' debug-key, chosen from
the limited remaining
>> > keys only due to its proximity to ''q'', that
dumps the console ring to
>> > configured console devices. It''s useful to for tracking
down how an
>> > unresponsive system got into a broken state via serial console.
>> 
>> Why wouldn''t everything on the ring already have been sent to
the configured
>> console devices?
> 
> It would have been. But what if no one was looking, or dom0 cleared
> the console, or placed it into a scrolling mode that prevents a
> virtual terminal from placing output into a scrollback buffer. With
> this patch, you can connect to a serial console after a machine
> becomes unresponsive and gather clues on what went wrong.
That''s an operator error then. I completely agree with Keir
that what you propose is pretty questionable (and you provide
one argument against it already - the shortage of remaining
keys).

Jan

Jan Beulich writes ("Re: [Xen-devel] [PATCH] xen/console: introduce a
''w'' debug-key that dumps the console
ring"):
> On 29.09.12 at 20:49, Matt Wilson <msw@amazon.com> wrote:
> > It would have been. But what if no one was looking, or dom0 cleared
> > the console, or placed it into a scrolling mode that prevents a
> > virtual terminal from placing output into a scrollback buffer. With
> > this patch, you can connect to a serial console after a machine
> > becomes unresponsive and gather clues on what went wrong.
> 
> That''s an operator error then.
Speaking as someone who does some sysadmin in their spare time, I
think this would be a very useful feature.  It is often the case that,
best intentions notwithstanding, the arrangements for collecting
serial output fail.  Many serial concentrators and ipkvms are very
bad.
> I completely agree with Keir that what you propose is pretty
> questionable (and you provide one argument against it already - the
> shortage of remaining keys).
If necessary we can have multi-keystroke keys, surely.  We should
start with assigning a prefix punctuation character which will give us
another 52 letters to play with.

Ian.

On 01/10/2012 16:18, "Ian Jackson" <Ian.Jackson@eu.citrix.com>
wrote:
> Jan Beulich writes ("Re: [Xen-devel] [PATCH] xen/console: introduce a
''w''
> debug-key that dumps the console ring"):
>> On 29.09.12 at 20:49, Matt Wilson <msw@amazon.com> wrote:
>>> It would have been. But what if no one was looking, or dom0 cleared
>>> the console, or placed it into a scrolling mode that prevents a
>>> virtual terminal from placing output into a scrollback buffer. With
>>> this patch, you can connect to a serial console after a machine
>>> becomes unresponsive and gather clues on what went wrong.
>> 
>> That''s an operator error then.
> 
> Speaking as someone who does some sysadmin in their spare time, I
> think this would be a very useful feature.  It is often the case that,
> best intentions notwithstanding, the arrangements for collecting
> serial output fail.  Many serial concentrators and ipkvms are very
> bad.
Yes, I must say that I found Matt''s explanation quite compelling too.
Clearly a large user of Xen is finding it useful, or it would not have been
proposed. ;)

So I will probably take the patch, and indeed running out of keys is a poor
reason for rejecting, as we can switch to two letter combos or a really
simple shell for extended commands.

 -- Keir
>> I completely agree with Keir that what you propose is pretty
>> questionable (and you provide one argument against it already - the
>> shortage of remaining keys).
> 
> If necessary we can have multi-keystroke keys, surely.  We should
> start with assigning a prefix punctuation character which will give us
> another 52 letters to play with.
> 
> Ian.