murble-xendev@yuri.org.uk
2004-Jul-11 23:24 UTC
[Xen-devel] Xenlinux oops and crashes with nbd, Xen no longer boots
Hi, When I start a new domain using an nbd exported device into domain0, xenlinux Oops and crashes. Nbd otherwise works fine in domain0 I can mount filesystems on it, dd them all, export them from domain0 to a new domain via nfs from domain0 also works. This is easily reproducable, with somthing like disk = [ ''phy:/dev/xen-uml0,sda1,w'' ] Where xen-uml0 is the nbd exported device. With Changeset 1.1063 Xen crashes at start up only displaying the figlet banner + (Xen) Unknown Interrupt. Hardware is processor : 0 vendor_id : AuthenticAMD cpu family : 6 model : 3 model name : AMD Duron(tm) Processor stepping : 1 cpu MHz : 697.541 cache size : 64 KB fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 1 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr syscall mmxext 3dnowext 3dnow bogomips : 6973.03 -- bill boughton
Ian Pratt
2004-Jul-12 18:27 UTC
Re: [Xen-devel] Xenlinux oops and crashes with nbd, Xen no longer boots
> When I start a new domain using an nbd exported device into domain0, > xenlinux Oops and crashes. Nbd otherwise works fine in domain0 I > can mount filesystems on it, dd them all, export them from domain0 > to a new domain via nfs from domain0 also works.You mean the domain0 oops''es, right? There was a fix checked in earlier today that might be relevant to this. Please can you try it out.> With Changeset 1.1063 Xen crashes at start up only displaying the > figlet banner + (Xen) Unknown Interrupt.This bug was introduced as part on an x86_64 checkin, and was due to a gcc version incompatibility, and is believed fixed. Ian ------------------------------------------------------- This SF.Net email sponsored by Black Hat Briefings & Training. Attend Black Hat Briefings & Training, Las Vegas July 24-29 - digital self defense, top technical experts, no vendor pitches, unmatched networking opportunities. Visit www.blackhat.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Keir Fraser
2004-Jul-12 21:06 UTC
Re: [Xen-devel] Xenlinux oops and crashes with nbd, Xen no longer boots
> > Hi, > > > When I start a new domain using an nbd exported device into domain0, > xenlinux Oops and crashes. Nbd otherwise works fine in domain0 I > can mount filesystems on it, dd them all, export them from domain0 > to a new domain via nfs from domain0 also works. > > This is easily reproducable, with somthing like > > disk = [ ''phy:/dev/xen-uml0,sda1,w'' ] > > Where xen-uml0 is the nbd exported device.>From what I can figure from the backtrace, it looks like the problemis an assertion failure due to a non-atomic memory allocation request while in interrupt context. Unfortunately the backtrace is a real mess. I might have better luck deciphering it if you can provide a link to your ''vmlinux'' file. Even better, recompile your DOM0 kernel with arch/xen/kernel/traps.c modified with ''kstack_depth_to_print'' changed to a larger value like 240. This will give a fuller stack backtrace (it''s currently less useful because it''s truncated). Then provide the backtrace and the vmlinux file.> With Changeset 1.1063 Xen crashes at start up only displaying the > figlet banner + (Xen) Unknown Interrupt.I fixed this earlier today. -- Keir ------------------------------------------------------- This SF.Net email sponsored by Black Hat Briefings & Training. Attend Black Hat Briefings & Training, Las Vegas July 24-29 - digital self defense, top technical experts, no vendor pitches, unmatched networking opportunities. Visit www.blackhat.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Keir Fraser
2004-Jul-13 07:12 UTC
Re: [Xen-devel] Xenlinux oops and crashes with nbd, Xen no longer boots
Okay, detangling the backtrace.... Entry EIP Function Caller''s return EIP --------- -------- ------------------- 0xc00c1220 alloc_skb(1656,GFP_NOIO) 0xc00ddd58 0xc00dd4b0 tcp_sendmsg 0xc00fa9a1 0xc00fa960 inet_sendmsg 0xc00bde23 0xc00bddb0 sock_sendmsg 0xc3cad25f unknown nbd_xmit 0xc3cad511 unknown nbd_send_req 0xc3cad91c unknown do_nbd_request 0xc008fc46 0xc008fbe0 generic_unplug_device 0xc000c342 0xc000c2e0 __run_task_queue n/a (tail call) 0xc00b8f20 io_schedule 0xc000c0a8 0xc000c050 tasklet_action 0xc000be9e 0xc000bdc0 do_softirq 0xc006f2c0 0xc006f220 do_IRQ(133,-) 0xc007395e 0xc00738c0 evtchn_do_upcall 0xc006dcd7 0xc006dca4 hypervisor_callback n/a (async callback) I see the problem. Well hidden because the key function (io_schedule) ends in a tail call. The problem is that we are scheduling I/O requests in interrupt context rather than deferring to a process context --- a known issue that hasn''t bitten until now. I''ll have to think about a fix. -- Keir> Down not across > ksymoops 2.4.9 on i686 2.4.26-xen0. Options used > -v ./vmlinux (specified) > -k /proc/ksyms (default) > -l /proc/modules (default) > -o /lib/modules/2.4.26-xen0/ (default) > -M (specified) > > invalid operand: 0000 > CPU: 0 > EIP: 0819:[<c00c133b>] Not tainted > Using defaults from ksymoops -t elf32-i386 -a i386 > EFLAGS: 00213286 > eax: 0000003a ebx: c048f040 ecx: c0178000 edx: 00000000 > esi: 00000030 edi: c048f160 ebp: c0179cd0 esp: c0179cc0 > ds: 0821 es: 0821 ss: 0821 > Process swapper (pid: 0, stackpage=c0179000)<1> > Stack: c012c920 c00ddd58 c048f040 c048f040 c0179d48 c00ddd58 00000678 00000030 > 00000004 00000008 0000000e c27f9b24 c0179d18 c00cc1cc c05dac00 c05dac00 > c05d000e c0179d28 c002097c c048f098 c0179ea8 0000001c 00000000 000005a8 > 00004000 00000000 c048f160 c0179ddc c37fe4e0 c05dac00 7fffffff c048f040 > c0179e5c c0179d78 c0179d60 c00fa9a1 c048f040 c0179e5c 0000001c 00000000 > c0179da4 c00bde23 c0d7bf0c c0179e5c 0000001c c0179d78 00000000 00000000 > 00000000 00000000 00000000 c37fe3d8 c218e000 c218ede8 c0179ea8 c0d7bf0c > 0000001c c0179e88 c3cad25f c0d7bf0c c0179e5c 0000001c c1eaa538 c35581dc > c1d13960 00000000 00000000 00000000 ffffffff c0179ea8 0000001c c1eaa538 > 00000000 00000030 c35581dc c0179e10 c00bbe62 c01c5444 c35581dc c250e000 > 00000000 c1d13800 c1eaa538 10000000 c0179e28 c00c550f c1eaa538 c1d13800 > 00000040 c1eaa538 c0179e6c c3c0fa28 c1eaa538 00000010 00000000 00000006 > c0179e90 0000003a c22997ac 00000002 c0179e6c 00000000 c0179e6c 00000000 > 00000000 c0179dd4 00000001 00000000 00000000 00004000 00000002 c3caf440 > c0d7bf0c c3caf420 c0179ed4 c3cad511 00000001 c0d7bf0c c0179ea8 0000001c > 00000000 c0d7bf0c 13956025 00000000 c2478c64 c0109f58 00000000 00040000 > 00040000 c010b1d4 c2478c64 c3caf420 00000000 c0179ef0 c3cad91c c3caf420 > c2478c64 00000000 c0179f0c fffffff7 c0179f00 c008fc46 c01ab6a4 c0179f0c > c0179f1c c000c342 c01ab6a4 c01ab704 c01ab704 00000000 c01881d8 c0179f2c > c000c0a8 c0147510 00000001 c0179f48 c000be9e c01881d8 00000001 c018a400 > c1d4bbec 00002140 c0179f68 c006f2c0 00000085 c0179f9c c1d4bbec 00000000 > 00000000 fbff9000 c0179f90 c007395e 00000085 c0179f9c 00000001 00000000 > c0179f9c c0178000 c0178000 00000006 c0179fe4 c006dcd7 c0179f9c 00000001 > 39854980 38ecb300 c0178000 00000006 c0179fe4 00000006 00000821 00000821 > 00000006 c006bcc2 00000819 00203246 c0179fe4 c0178000 00000000 c01e2200 > c01983a0 c0179ffc c017a57c c0116f40 c01986a0 00000000 c01985a0 00000000 > Call Trace: [<c00ddd58>] [<c00ddd58>] [<c00cc1cc>] [<c002097c>] [<c00fa9a1>] > [<c00bde23>] [<c3cad25f>] [<c00bbe62>] [<c00c550f>] [<c3c0fa28>] [<c3caf440>] > [<c3caf420>] [<c3cad511>] [<c0109f58>] [<c010b1d4>] [<c3caf420>] [<c3cad91c>] > [<c3caf420>] [<c008fc46>] [<c000c342>] [<c000c0a8>] [<c000be9e>] [<c006f2c0>] > [<c007395e>] [<c006dcd7>] [<c006bcc2>] > <0>Kernel panic: Aiee, killing interrupt handler! > Warning (Oops_read): Code line not seen, dumping what data is available > > > >>EIP; c00c133b <alloc_skb+11b/130> <====> > >>ebx; c048f040 <_end+2bd130/3a2e170> > >>ecx; c0178000 <init_task_union+0/2000> > >>edi; c048f160 <_end+2bd250/3a2e170> > >>ebp; c0179cd0 <init_task_union+1cd0/2000> > >>esp; c0179cc0 <init_task_union+1cc0/2000> > > Trace; c00ddd58 <tcp_sendmsg+8a8/f40> > Trace; c00ddd58 <tcp_sendmsg+8a8/f40> > Trace; c00cc1cc <rtnetlink_put_metrics+9c/150> > Trace; c002097c <kmem_cache_free_one+ec/200> > Trace; c00fa9a1 <inet_sendmsg+41/50> > Trace; c00bde23 <sock_sendmsg+73/b0> > Trace; c3cad25f <[nbd]nbd_xmit+11f/2e0> > Trace; c00bbe62 <netif_be_start_xmit+122/180> > Trace; c00c550f <dev_queue_xmit+9f/270> > Trace; c3c0fa28 <[ipv6]ndisc_send_rs+1a8/230> > Trace; c3caf440 <[nbd].data.end+a59/3699> > Trace; c3caf420 <[nbd].data.end+a39/3699> > Trace; c3cad511 <[nbd]nbd_send_req+f1/180> > Trace; c0109f58 <br_config_bpdu_generation+38/40> > Trace; c010b1d4 <br_hello_timer_expired+14/30> > Trace; c3caf420 <[nbd].data.end+a39/3699> > Trace; c3cad91c <[nbd]do_nbd_request+fc/1d0> > Trace; c3caf420 <[nbd].data.end+a39/3699> > Trace; c008fc46 <generic_unplug_device+66/70> > Trace; c000c342 <__run_task_queue+62/80> > Trace; c000c0a8 <tasklet_action+58/b0> > Trace; c000be9e <do_softirq+de/100> > Trace; c006f2c0 <do_IRQ+a0/b0> > Trace; c007395e <evtchn_do_upcall+9e/100> > Trace; c006dcd7 <hypervisor_callback+33/49> > Trace; c006bcc2 <cpu_idle+62/c0> > > > 1 warning issued. Results may not be reliable.-=- MIME -=- --u3/rZRmxL6MmkK24 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Mon, Jul 12, 2004 at 10:06:55PM +0100, Keir Fraser wrote:> Even better, recompile your DOM0 kernel with arch/xen/kernel/traps.c > modified with ''kstack_depth_to_print'' changed to a larger value like > 240. This will give a fuller stack backtrace (it''s currently less > useful because it''s truncated). Then provide the backtrace and the > vmlinux file.OK I''ve done that the vmlinux file is at http://www.yuri.org.uk/~murble/vmlinux-xen New Oops file should be attached. cheers bill -- Bill Boughton <murble(at)@yuri.org.uk> <bill@(at)xencat.murble.org.uk> Down not across --u3/rZRmxL6MmkK24 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=xen0-oops ksymoops 2.4.9 on i686 2.4.26-xen0. Options used -v ./vmlinux (specified) -k /proc/ksyms (default) -l /proc/modules (default) -o /lib/modules/2.4.26-xen0/ (default) -M (specified) invalid operand: 0000 CPU: 0 EIP: 0819:[<c00c133b>] Not tainted Using defaults from ksymoops -t elf32-i386 -a i386 EFLAGS: 00213286 eax: 0000003a ebx: c048f040 ecx: c0178000 edx: 00000000 esi: 00000030 edi: c048f160 ebp: c0179cd0 esp: c0179cc0 ds: 0821 es: 0821 ss: 0821 Process swapper (pid: 0, stackpage=c0179000)<1> Stack: c012c920 c00ddd58 c048f040 c048f040 c0179d48 c00ddd58 00000678 00000030 00000004 00000008 0000000e c27f9b24 c0179d18 c00cc1cc c05dac00 c05dac00 c05d000e c0179d28 c002097c c048f098 c0179ea8 0000001c 00000000 000005a8 00004000 00000000 c048f160 c0179ddc c37fe4e0 c05dac00 7fffffff c048f040 c0179e5c c0179d78 c0179d60 c00fa9a1 c048f040 c0179e5c 0000001c 00000000 c0179da4 c00bde23 c0d7bf0c c0179e5c 0000001c c0179d78 00000000 00000000 00000000 00000000 00000000 c37fe3d8 c218e000 c218ede8 c0179ea8 c0d7bf0c 0000001c c0179e88 c3cad25f c0d7bf0c c0179e5c 0000001c c1eaa538 c35581dc c1d13960 00000000 00000000 00000000 ffffffff c0179ea8 0000001c c1eaa538 00000000 00000030 c35581dc c0179e10 c00bbe62 c01c5444 c35581dc c250e000 00000000 c1d13800 c1eaa538 10000000 c0179e28 c00c550f c1eaa538 c1d13800 00000040 c1eaa538 c0179e6c c3c0fa28 c1eaa538 00000010 00000000 00000006 c0179e90 0000003a c22997ac 00000002 c0179e6c 00000000 c0179e6c 00000000 00000000 c0179dd4 00000001 00000000 00000000 00004000 00000002 c3caf440 c0d7bf0c c3caf420 c0179ed4 c3cad511 00000001 c0d7bf0c c0179ea8 0000001c 00000000 c0d7bf0c 13956025 00000000 c2478c64 c0109f58 00000000 00040000 00040000 c010b1d4 c2478c64 c3caf420 00000000 c0179ef0 c3cad91c c3caf420 c2478c64 00000000 c0179f0c fffffff7 c0179f00 c008fc46 c01ab6a4 c0179f0c c0179f1c c000c342 c01ab6a4 c01ab704 c01ab704 00000000 c01881d8 c0179f2c c000c0a8 c0147510 00000001 c0179f48 c000be9e c01881d8 00000001 c018a400 c1d4bbec 00002140 c0179f68 c006f2c0 00000085 c0179f9c c1d4bbec 00000000 00000000 fbff9000 c0179f90 c007395e 00000085 c0179f9c 00000001 00000000 c0179f9c c0178000 c0178000 00000006 c0179fe4 c006dcd7 c0179f9c 00000001 39854980 38ecb300 c0178000 00000006 c0179fe4 00000006 00000821 00000821 00000006 c006bcc2 00000819 00203246 c0179fe4 c0178000 00000000 c01e2200 c01983a0 c0179ffc c017a57c c0116f40 c01986a0 00000000 c01985a0 00000000 Call Trace: [<c00ddd58>] [<c00ddd58>] [<c00cc1cc>] [<c002097c>] [<c00fa9a1>] [<c00bde23>] [<c3cad25f>] [<c00bbe62>] [<c00c550f>] [<c3c0fa28>] [<c3caf440>] [<c3caf420>] [<c3cad511>] [<c0109f58>] [<c010b1d4>] [<c3caf420>] [<c3cad91c>] [<c3caf420>] [<c008fc46>] [<c000c342>] [<c000c0a8>] [<c000be9e>] [<c006f2c0>] [<c007395e>] [<c006dcd7>] [<c006bcc2>] <0>Kernel panic: Aiee, killing interrupt handler! Warning (Oops_read): Code line not seen, dumping what data is available>>EIP; c00c133b <alloc_skb+11b/130> <==== >>ebx; c048f040 <_end+2bd130/3a2e170> >>ecx; c0178000 <init_task_union+0/2000> >>edi; c048f160 <_end+2bd250/3a2e170> >>ebp; c0179cd0 <init_task_union+1cd0/2000> >>esp; c0179cc0 <init_task_union+1cc0/2000>Trace; c00ddd58 <tcp_sendmsg+8a8/f40> Trace; c00ddd58 <tcp_sendmsg+8a8/f40> Trace; c00cc1cc <rtnetlink_put_metrics+9c/150> Trace; c002097c <kmem_cache_free_one+ec/200> Trace; c00fa9a1 <inet_sendmsg+41/50> Trace; c00bde23 <sock_sendmsg+73/b0> Trace; c3cad25f <[nbd]nbd_xmit+11f/2e0> Trace; c00bbe62 <netif_be_start_xmit+122/180> Trace; c00c550f <dev_queue_xmit+9f/270> Trace; c3c0fa28 <[ipv6]ndisc_send_rs+1a8/230> Trace; c3caf440 <[nbd].data.end+a59/3699> Trace; c3caf420 <[nbd].data.end+a39/3699> Trace; c3cad511 <[nbd]nbd_send_req+f1/180> Trace; c0109f58 <br_config_bpdu_generation+38/40> Trace; c010b1d4 <br_hello_timer_expired+14/30> Trace; c3caf420 <[nbd].data.end+a39/3699> Trace; c3cad91c <[nbd]do_nbd_request+fc/1d0> Trace; c3caf420 <[nbd].data.end+a39/3699> Trace; c008fc46 <generic_unplug_device+66/70> Trace; c000c342 <__run_task_queue+62/80> Trace; c000c0a8 <tasklet_action+58/b0> Trace; c000be9e <do_softirq+de/100> Trace; c006f2c0 <do_IRQ+a0/b0> Trace; c007395e <evtchn_do_upcall+9e/100> Trace; c006dcd7 <hypervisor_callback+33/49> Trace; c006bcc2 <cpu_idle+62/c0> 1 warning issued. Results may not be reliable. --u3/rZRmxL6MmkK24-- ------------------------------------------------------- This SF.Net email sponsored by Black Hat Briefings & Training. Attend Black Hat Briefings & Training, Las Vegas July 24-29 - digital self defense, top technical experts, no vendor pitches, unmatched networking opportunities. Visit www.blackhat.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Keir Fraser
2004-Jul-13 15:29 UTC
Re: [Xen-devel] Xenlinux oops and crashes with nbd, Xen no longer boots
Now fixed. You''ve been able to hit this more reliably than anyone else, so I''ll be interested to hear confirmation that the problem really has now gone away. -- Keir> > Okay, detangling the backtrace.... > > Entry EIP Function Caller''s return EIP > --------- -------- ------------------- > 0xc00c1220 alloc_skb(1656,GFP_NOIO) 0xc00ddd58 > 0xc00dd4b0 tcp_sendmsg 0xc00fa9a1 > 0xc00fa960 inet_sendmsg 0xc00bde23 > 0xc00bddb0 sock_sendmsg 0xc3cad25f > unknown nbd_xmit 0xc3cad511 > unknown nbd_send_req 0xc3cad91c > unknown do_nbd_request 0xc008fc46 > 0xc008fbe0 generic_unplug_device 0xc000c342 > 0xc000c2e0 __run_task_queue n/a (tail call) > 0xc00b8f20 io_schedule 0xc000c0a8 > 0xc000c050 tasklet_action 0xc000be9e > 0xc000bdc0 do_softirq 0xc006f2c0 > 0xc006f220 do_IRQ(133,-) 0xc007395e > 0xc00738c0 evtchn_do_upcall 0xc006dcd7 > 0xc006dca4 hypervisor_callback n/a (async callback) > > I see the problem. Well hidden because the key function (io_schedule) > ends in a tail call. The problem is that we are scheduling I/O > requests in interrupt context rather than deferring to a process > context --- a known issue that hasn''t bitten until now. I''ll have to > think about a fix. > > -- Keir------------------------------------------------------- This SF.Net email sponsored by Black Hat Briefings & Training. Attend Black Hat Briefings & Training, Las Vegas July 24-29 - digital self defense, top technical experts, no vendor pitches, unmatched networking opportunities. Visit www.blackhat.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Wm Boughton
2004-Jul-13 16:36 UTC
Re: [Xen-devel] Xenlinux oops and crashes with nbd, Xen no longer boots
On Tue, Jul 13, 2004 at 04:29:00PM +0100, Keir Fraser wrote:> > Now fixed. You''ve been able to hit this more reliably than anyone > else, so I''ll be interested to hear confirmation that the problem > really has now gone away.I''ve not been abel to hit it and am now happily booting XenLinux instances over nbd. thanks bill ------------------------------------------------------- This SF.Net email sponsored by Black Hat Briefings & Training. Attend Black Hat Briefings & Training, Las Vegas July 24-29 - digital self defense, top technical experts, no vendor pitches, unmatched networking opportunities. Visit www.blackhat.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Steve Traugott
2004-Jul-23 06:43 UTC
Re: [Xen-devel] Xenlinux oops and crashes with nbd, Xen no longer boots
Hi Bill, On Tue, Jul 13, 2004 at 05:36:38PM +0100, Wm Boughton wrote:> I''ve not been abel to hit it and am now happily booting > XenLinux instances over nbd.Which NBD are you using? The kernel NBD, ENBD, or GNBD? Steve -- Stephen G. Traugott (KG6HDQ) UNIX/Linux Infrastructure Architect, TerraLuna LLC stevegt@TerraLuna.Org http://www.stevegt.com -- http://Infrastructures.Org ------------------------------------------------------- This SF.Net email is sponsored by BEA Weblogic Workshop FREE Java Enterprise J2EE developer tools! Get your free copy of BEA WebLogic Workshop 8.1 today. http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
hi there, How to debug xen? just use the nsplitd as XenDebugger-HOWTO said? It seems quite tricky. I believe lots of people are doing development on xen, so usually how to debug it? Thanks Yan ------------------------------------------------------- This SF.Net email is sponsored by BEA Weblogic Workshop FREE Java Enterprise J2EE developer tools! Get your free copy of BEA WebLogic Workshop 8.1 today. http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
" How to debug xen? just use the nsplitd as XenDebugger-HOWTO said? It
seems
" quite tricky. I believe lots of people are doing development on xen, so
" usually how to debug it?
This is how I debug xen. The starting point is reading
xeno-unstable.bk/docs/HOWTOs/XenDebugger-HOWTO
I do not use nsplitd.
First, build debug kernels.
For xen, in the xeno-unstable.bk/xen directory add this to arch/x86/Rules.mk
CFLAGS += -g
and build xen saying
make debugger=y
For xenolinux, in xeno-unstable.bk/linux-2.4.26-xen0
turn on CONFIG_FRAME_POINTER (in the Kernel Hacking section of
menuconfig or xconfig)
in Makefile put -g into CFLAGS_KERNEL (line 52)
CFLAGS_KERNEL = -g
and build xenolinux with
make ARCH=xen bzImage
Second, find a target machine with some serial lines.
I prefer virtual machines and use VMware Workstation 4.5 on Linux (4.0
didn''t
work, btw). In the vmware Machine Settings, config both com1 and com2
to be named pipes with "This end is the client" and "The other
end is a
virtual machine". Run two socat jobs in the background
to convert the pipes to ptys:
socat UNIX-LISTEN:/tmp/com1pipe,unlink-early,fork pty,link=/tmp/com1 &
socat UNIX-LISTEN:/tmp/com2pipe,unlink-early,fork pty,link=/tmp/com2 &
Third, boot the target machine and connect the debugger.
Put the debug xen and xenolinux on the machine. In the grub menu.lst
file, add the serial lines and pdb to the xen command line:
kernel /boot/xen.gz.debug dom0_mem=100000 ifname=eth0 com1=9600,8n1
com2=9600,8n1 pdb=com2
Boot it up and connect to the console. In my vmware setup, I connect
to com1 from the machine hosting VMware with
kermit -c -l /tmp/com1
Then on com1 type ^A^A^A to make xen listen to console cmds.
At this point type ''h'' and see if the ''D''
command is listed. If not,
pdb is missing so make sure the xen being run was built with debugger=y.
Hit ''D'' on the console to make xen drop into the debugger.
>From the machine listening to the serial lines, connect to the target
machine
with gdb and the xen-syms file and check if was built with -g by trying to
list a function:
$ gdb xeno-unstable.bk/xen/xen-syms
(gdb) list do_dom0_op
26 extern unsigned int alloc_new_dom_mem(struct domain *,
unsigned int);
27 extern long arch_do_dom0_op(dom0_op_t *op, dom0_op_t
*u_dom0_op);
28 extern void arch_getdomaininfo_ctxt(struct domain *,
full_execution_context_t *);
29
30 long do_dom0_op(dom0_op_t *u_dom0_op)
31 {
32 long ret = 0;
33 dom0_op_t curop, *op = &curop;
34
35 if ( !IS_PRIV(current) )
(gdb) target remote /tmp/com2
Now you can debug xen itself. Note that the breakpoint is in xenolinux
so gdb will be cryptic about the current function and stacktrace. Put a
breakpoint in xen somewhere and continue to get gdb to a function is can
decode.
To debug xenolinux, drop to the debugger:
(gdb) set pdb_ctx.domain=0
(gdb) set pdb_ctx.valid=1
(gdb) add-symbol-file xeno-unstable.bk/linux-2.4.26-xen0/vmlinux
Now you can debug vmlinux itself (except where the xen and linux
symbols clash).
-------------------------------------------------------
This SF.Net email is sponsored by BEA Weblogic Workshop
FREE Java Enterprise J2EE developer tools!
Get your free copy of BEA WebLogic Workshop 8.1 today.
http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel
One non-obvious thing that I didn''t see in the docs is that if you use multiple nsplitds they have to be spaced 4 ports apart not 2. -Kip On Fri, 23 Jul 2004, David Becker wrote:> > " How to debug xen? just use the nsplitd as XenDebugger-HOWTO said? It seems > " quite tricky. I believe lots of people are doing development on xen, so > " usually how to debug it? > > This is how I debug xen. The starting point is reading > xeno-unstable.bk/docs/HOWTOs/XenDebugger-HOWTO > I do not use nsplitd. > > First, build debug kernels. > For xen, in the xeno-unstable.bk/xen directory add this to arch/x86/Rules.mk > CFLAGS += -g > and build xen saying > make debugger=y > For xenolinux, in xeno-unstable.bk/linux-2.4.26-xen0 > turn on CONFIG_FRAME_POINTER (in the Kernel Hacking section of > menuconfig or xconfig) > in Makefile put -g into CFLAGS_KERNEL (line 52) > CFLAGS_KERNEL = -g > and build xenolinux with > make ARCH=xen bzImage > > > Second, find a target machine with some serial lines. > I prefer virtual machines and use VMware Workstation 4.5 on Linux (4.0 didn''t > work, btw). In the vmware Machine Settings, config both com1 and com2 > to be named pipes with "This end is the client" and "The other end is a > virtual machine". Run two socat jobs in the background > to convert the pipes to ptys: > socat UNIX-LISTEN:/tmp/com1pipe,unlink-early,fork pty,link=/tmp/com1 & > socat UNIX-LISTEN:/tmp/com2pipe,unlink-early,fork pty,link=/tmp/com2 & > > > Third, boot the target machine and connect the debugger. > Put the debug xen and xenolinux on the machine. In the grub menu.lst > file, add the serial lines and pdb to the xen command line: > kernel /boot/xen.gz.debug dom0_mem=100000 ifname=eth0 com1=9600,8n1 com2=9600,8n1 pdb=com2 > Boot it up and connect to the console. In my vmware setup, I connect > to com1 from the machine hosting VMware with > kermit -c -l /tmp/com1 > Then on com1 type ^A^A^A to make xen listen to console cmds. > At this point type ''h'' and see if the ''D'' command is listed. If not, > pdb is missing so make sure the xen being run was built with debugger=y. > Hit ''D'' on the console to make xen drop into the debugger. > > >From the machine listening to the serial lines, connect to the target machine > with gdb and the xen-syms file and check if was built with -g by trying to > list a function: > $ gdb xeno-unstable.bk/xen/xen-syms > (gdb) list do_dom0_op > 26 extern unsigned int alloc_new_dom_mem(struct domain *, > unsigned int); > 27 extern long arch_do_dom0_op(dom0_op_t *op, dom0_op_t > *u_dom0_op); > 28 extern void arch_getdomaininfo_ctxt(struct domain *, > full_execution_context_t *); > 29 > 30 long do_dom0_op(dom0_op_t *u_dom0_op) > 31 { > 32 long ret = 0; > 33 dom0_op_t curop, *op = &curop; > 34 > 35 if ( !IS_PRIV(current) ) > (gdb) target remote /tmp/com2 > Now you can debug xen itself. Note that the breakpoint is in xenolinux > so gdb will be cryptic about the current function and stacktrace. Put a > breakpoint in xen somewhere and continue to get gdb to a function is can > decode. > > To debug xenolinux, drop to the debugger: > (gdb) set pdb_ctx.domain=0 > (gdb) set pdb_ctx.valid=1 > (gdb) add-symbol-file xeno-unstable.bk/linux-2.4.26-xen0/vmlinux > Now you can debug vmlinux itself (except where the xen and linux > symbols clash). > > > > ------------------------------------------------------- > This SF.Net email is sponsored by BEA Weblogic Workshop > FREE Java Enterprise J2EE developer tools! > Get your free copy of BEA WebLogic Workshop 8.1 today. > http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xen-devel >------------------------------------------------------- This SF.Net email is sponsored by BEA Weblogic Workshop FREE Java Enterprise J2EE developer tools! Get your free copy of BEA WebLogic Workshop 8.1 today. http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
hi Thanks,but now I''m stuck. I wanna try nsplitd first. so I do it according to the Debugger-howto. I built xen and xeno with -g added and fomit-frame-pointer disabled. The problem is I''m really confused by the configuration of nsplitd. 1.it said set ''pdb=xxx''. How can I tell which port I''m connecting to? 2.in nsplitd configuration. what are the /etc/xinetd.d/nsplit and all the ''wcons00'', ''wanda remote console, stuff? I''m asuming it''s in configuration in their working machine. Then can anyone tell me what does this mean? The whole nsplitd configuration section,please let me know the details about it. I totally have no idea what''s going on there. For example, I have two machines, A.x.x.x, B.x.x.x, I''m running xen on machine A, using machine B to connect to the serial line. What should all the settings look like? btw, David, is the socat just this one: http://www.socat.org/? Thanks a lot! Q ----- Original Message ----- From: "Kip Macy" <kmacy@eventdriven.org> To: "David Becker" <becker@cs.duke.edu> Cc: "Yan Li" <yan_li00@hotmail.com>; <xen-devel@lists.sourceforge.net> Sent: Friday, July 23, 2004 8:08 AM Subject: Re: [Xen-devel] how to debug xen?> One non-obvious thing that I didn''t see in the docs is that if you use > multiple nsplitds they have to be spaced 4 ports apart not 2. > > -Kip > > > > On Fri, 23 Jul 2004, David Becker wrote: > > > > > " How to debug xen? just use the nsplitd as XenDebugger-HOWTO said? Itseems> > " quite tricky. I believe lots of people are doing development on xen,so> > " usually how to debug it? > > > > This is how I debug xen. The starting point is reading > > xeno-unstable.bk/docs/HOWTOs/XenDebugger-HOWTO > > I do not use nsplitd. > > > > First, build debug kernels. > > For xen, in the xeno-unstable.bk/xen directory add this toarch/x86/Rules.mk> > CFLAGS += -g > > and build xen saying > > make debugger=y > > For xenolinux, in xeno-unstable.bk/linux-2.4.26-xen0 > > turn on CONFIG_FRAME_POINTER (in the Kernel Hacking section of > > menuconfig or xconfig) > > in Makefile put -g into CFLAGS_KERNEL (line 52) > > CFLAGS_KERNEL = -g > > and build xenolinux with > > make ARCH=xen bzImage > > > > > > Second, find a target machine with some serial lines. > > I prefer virtual machines and use VMware Workstation 4.5 on Linux (4.0didn''t> > work, btw). In the vmware Machine Settings, config both com1 and com2 > > to be named pipes with "This end is the client" and "The other end is a > > virtual machine". Run two socat jobs in the background > > to convert the pipes to ptys: > > socat UNIX-LISTEN:/tmp/com1pipe,unlink-early,fork pty,link=/tmp/com1&> > socat UNIX-LISTEN:/tmp/com2pipe,unlink-early,fork pty,link=/tmp/com2&> > > > > > Third, boot the target machine and connect the debugger. > > Put the debug xen and xenolinux on the machine. In the grub menu.lst > > file, add the serial lines and pdb to the xen command line: > > kernel /boot/xen.gz.debug dom0_mem=100000 ifname=eth0 com1=9600,8n1com2=9600,8n1 pdb=com2> > Boot it up and connect to the console. In my vmware setup, I connect > > to com1 from the machine hosting VMware with > > kermit -c -l /tmp/com1 > > Then on com1 type ^A^A^A to make xen listen to console cmds. > > At this point type ''h'' and see if the ''D'' command is listed. If not, > > pdb is missing so make sure the xen being run was built with debugger=y. > > Hit ''D'' on the console to make xen drop into the debugger. > > > > >From the machine listening to the serial lines, connect to the targetmachine> > with gdb and the xen-syms file and check if was built with -g by tryingto> > list a function: > > $ gdb xeno-unstable.bk/xen/xen-syms > > (gdb) list do_dom0_op > > 26 extern unsigned int alloc_new_dom_mem(struct domain *, > > unsigned int); > > 27 extern long arch_do_dom0_op(dom0_op_t *op, dom0_op_t > > *u_dom0_op); > > 28 extern void arch_getdomaininfo_ctxt(struct domain *, > > full_execution_context_t *); > > 29 > > 30 long do_dom0_op(dom0_op_t *u_dom0_op) > > 31 { > > 32 long ret = 0; > > 33 dom0_op_t curop, *op = &curop; > > 34 > > 35 if ( !IS_PRIV(current) ) > > (gdb) target remote /tmp/com2 > > Now you can debug xen itself. Note that the breakpoint is in xenolinux > > so gdb will be cryptic about the current function and stacktrace. Put a > > breakpoint in xen somewhere and continue to get gdb to a function is can > > decode. > > > > To debug xenolinux, drop to the debugger: > > (gdb) set pdb_ctx.domain=0 > > (gdb) set pdb_ctx.valid=1 > > (gdb) add-symbol-file xeno-unstable.bk/linux-2.4.26-xen0/vmlinux > > Now you can debug vmlinux itself (except where the xen and linux > > symbols clash). > > > > > > > > ------------------------------------------------------- > > This SF.Net email is sponsored by BEA Weblogic Workshop > > FREE Java Enterprise J2EE developer tools! > > Get your free copy of BEA WebLogic Workshop 8.1 today. > > http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/xen-devel > > >------------------------------------------------------- This SF.Net email is sponsored by OSTG. Have you noticed the changes on Linux.com, ITManagersJournal and NewsForge in the past few weeks? Now, one more big change to announce. We are now OSTG- Open Source Technology Group. Come see the changes on the new OSTG site. www.ostg.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel