Randy Thelen
2006-May-26 00:39 UTC
[Xen-devel] FreeBSD boot loader on VT-x based Xen system
Folks -- I''m trying to run FreeBSD 6.0 on a VT-x based Xen system. I''m experiencing a problem where the boot loader is crashing before the FreeBSD kernel (proper) is loaded. The sequence of operations is that the VMX domain is constructed, the BIOS is executed, GRUB loads, I type the instructions to load FreeBSD (a boot loader) and that code begins to execute and then crashes. Here''s a console log that we captured:> GNU GRUB version 0.95 (638K lower / 261112K upper memory) > [ Minimal BASH-like line editing is supported. For the first word, > TAB > lists possible command completions. Anywhere else TAB lists the > possible > completions of a device/filename.] > > grub> root (hd0,1,a) > Filesystem type is ufs2, partition type 0xa5 > > grub> kernel /boot/loader > [FreeBSD-a.out, loadaddr=0x200000, text=0x1000, data=0x32000, > bss=0x0, entry =0x200000] > > grub> boot > BTX loader 1.00 Starting in protected mode (base mem=9fc00) > Arguments passed (esp=67e88): > <howto=80000000 bootdev=a0300000 junk=0 0 0 bootinfo=67ec4> > Relocated bootinfo (size=48) to 9fbb8 > Relocated arguments (size=18) to 9fba0 > BTX version is 1.01 > Relocated kernel (size=780) to 9000 > Client base address is a000 > Client format is ELF > text segment: offset=1000 vaddr=0 filesz=2a3b0 memsz=2a3b0 > data segment: offset=2c000 vaddr=2c000 filesz=53cc memsz=b4c0 > Loading complete > > int=0000000d err=00000000 efl=00010006 eip=000090ac > eax=00094c00 ebx=00202820 ecx=00000033 edx=0000a000 > esi=00209701 edi=00001f98 ebp=0000a000 esp=000017f0 > cs=0008 ds=0000 es=0000 fs=0020 gs=0020 ss=0010 > cs:eip=ff 35 0c 90 00 00 51 51-51 51 52 b1 07 6a 00 e2 > fc 61 07 1f 0f a1 0f a9-cf fa bc 00 18 00 00 0f > ss:esp=2b 00 00 00 02 02 00 00-00 4c 09 00 33 00 00 00 > 0a 69 6e 74 3d 30 30 30-30 30 30 30 64 20 20 65 > BTX haltedThe instructions that were being executed can be found, in a FreeBSD source tree, at: /usr/src/sys/boot/i386/btx/btx/btx.S Here they are for your perusal: .code32 init.8: xorl %ecx,%ecx # Zero movb $SEL_SDATA,%cl # To 32-bit movw %cx,%ss # stack /* * Launch user task. */ movb $SEL_TSS,%cl # Set task ltr %cx # register movl $MEM_USR,%edx # User base address movzwl %ss:BDA_MEM,%eax # Get free memory shll $0xa,%eax # To bytes subl $0x1000,%eax # Less arg space subl %edx,%eax # Less base movb $SEL_UDATA,%cl # User data selector pushl %ecx # Set SS pushl %eax # Set ESP push $0x202 # Set flags (IF set) push $SEL_UCODE # Set CS pushl btx_hdr+0xc # Set EIP The error occurs with the last instruction shown above: pushl btx_hdr +0xc. Here''s a disassembly of those instructions: 0000907f (02) 31c9 XOR ECX, ECX 00009081 (02) b1 10 MOV CL, 0x10 00009083 (02) 8ed1 MOV SS, CX 00009085 (02) b1 38 MOV CL, 0x38 00009087 (03) 0f00d9 LTR CX 0000908a (05) ba 00a00000 MOV EDX, 0xa000 0000908f (08) 36 0fb705 13040000 MOVZX EAX, SS:[0x413] 00009097 (03) c1e0 0a SHL EAX, 0xa 0000909a (05) 2d 00100000 SUB EAX, 0x1000 0000909f (02) 29d0 SUB EAX, EDX 000090a1 (02) b1 33 MOV CL, 0x33 000090a3 (01) 51 PUSH ECX 000090a4 (01) 50 PUSH EAX 000090a5 (05) 68 02020000 PUSH 0x202 000090aa (02) 6a 2b PUSH 0x2b 000090ac (06) ff35 0c000000 PUSH DWORD [0xc] And, we can see that the instruction stream at 90ac matches the instruction stream in the eip dump in the exception display. (Note that I''m disassembling the btx.o file, and so the linker has not fixed up the reference to btx_hdr+0xc to be 900c, instead we''re seeing just 0xc. Note too, that this explains why I believe I can replace the reference to btx_hdr+0xc with an explicit reference to the code segment register; see below.) My guess is that there''s a bug in VMXAssist. I''d like some corroboration of that point, if possible. In fact, because the code had long jumped to the above code sequence to enter 32bit mode, are we still in VMXAssist? Or have we entered into pure execution mode? It appears that the problem is based on an incorrect data segment register value. The code segment register contains 8, which points to a valid code segment descriptor. But, the ds register is 0, and that''s never good. How or why this code works on real hardware is undoubtedly a side effect of ... I dunno a bug, legacy, or a design that''s not clear to me. At any rate, If I replace the push instruction with: pushl %cs:btx_hdr+0xc referencing the code segment selector, then we get past the exception noted above and into our next problem. I''m just curious about this particular bug. Can somebody lend some insight or direction in tracking this down? I''m more than happy to do lots of leg work on tracking this down, but a little direction would go a long way to help. -- Randy _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Andy Clayton
2006-Jun-02 17:16 UTC
Re: [Xen-devel] FreeBSD boot loader on VT-x based Xen system
Randy Thelen wrote:> Folks -- > > I''m trying to run FreeBSD 6.0 on a VT-x based Xen system. I''m > experiencing a problem where the boot loader is crashing before the > FreeBSD kernel (proper) is loaded. >I don''t think I can help you much as far as fixing, but earlier I opened a bug report in Bugzilla regarding this issue at http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=622 I will try to help any way I can but I have next to no experience with assembly. Time to learn, perhaps. Good luck for now and ask if you would like some help (with something I can actually help with...), Andy Clayton _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Andy Clayton
2006-Jun-02 17:29 UTC
Re: [Xen-devel] FreeBSD boot loader on VT-x based Xen system
Andy Clayton wrote:> Randy Thelen wrote: >> Folks -- >> >> I''m trying to run FreeBSD 6.0 on a VT-x based Xen system. I''m >> experiencing a problem where the boot loader is crashing before the >> FreeBSD kernel (proper) is loaded. >> > > I don''t think I can help you much as far as fixing, but earlier I > opened a bug report in Bugzilla regarding this issue at > http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=622 > I will try to help any way I can but I have next to no experience with > assembly. Time to learn, perhaps. > > Good luck for now and ask if you would like some help (with something > I can actually help with...), > Andy Clayton >Hey there is a patch that was mailed to -devel by Kevin Tronkowski today called "[Xen-devel] [PATCH] Extensions to vmxassist to handle additional mode switch cases". It sounds promising. If I may quote: Attached is a patch which extends vmxassist to handle two cases related to mode switching found while experimenting with different boot loaders. The first case is use of the JMP instruction with memory location operands to complete a switch to protected mode. This patch adds emulation for this form of the JMP instruction to vmxassist. The second case is where boot loader code does not save/restore a non-zero SS register across a protected mode traversal. Zeroing the SS register in vmxassist results in all sorts of problems in the domU after returning back to real mode. This patch stores segment register values before entering protected mode and correctly restores the old values (instead of an incorrect zero value) when reentering real mode. The first case is what I was looking at. I''ll see if I can check this out shortly... Andy Clayton _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Andy Clayton wrote:> Andy Clayton wrote: >> Randy Thelen wrote: >>> Folks -- >>> >>> I''m trying to run FreeBSD 6.0 on a VT-x based Xen system. I''m >>> experiencing a problem where the boot loader is crashing before the >>> FreeBSD kernel (proper) is loaded. >>> >> >> I don''t think I can help you much as far as fixing, but earlier I >> opened a bug report in Bugzilla regarding this issue at >> http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=622 >> I will try to help any way I can but I have next to no experience >> with assembly. Time to learn, perhaps. >> >> Good luck for now and ask if you would like some help (with something >> I can actually help with...), >> Andy Clayton >> > Hey there is a patch that was mailed to -devel by Kevin Tronkowski > today called "[Xen-devel] [PATCH] Extensions to vmxassist to handle > additional mode switch cases". It sounds promising. >Wow, I really should catch up on all posts here before replying. Yeah... you already posted something. Ignore me for now, heh. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel