Hi Keir, Appears as if c/s 14346 has broken booting pae and 64bit HVM linux guests, on 64bit hv on AMD-V. Seems like grub can''t find a bootable device, and errors out. We haven''t tried VT boxes at this point. We just uncovered this issue and will investigate more. Looks like 32bit hypervisor works ok, but haven''t tested thoroughly. We''ll look at the patch, but if you have any thoughts. Appreciate taking a closer look at the modification. --Tom thomas.woller@amd.com AMD Corporation 5204 E. Ben White Blvd. UBC1 Austin, Texas 78741 +1-512-602-0059 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Initial email was incorrect. The issue is not with c/s 14346 but with
14345. c/s 14344 is able to boot pae/64b HVM AMD-V guests on AMD-V
without issue, while c/s 14345 fails.
[snip from 14345 changeset]
#endif
- /* Debugging info */
- printf("Booting from %x:%x\n", bootseg, bootip);
/* Jump to the boot vector */
ASM_START
[end code snippet]
Removing the printf causes pae and 64bit HVM AMD-V guests to fail to
boot on 64bit hypervisor.
If I add just the single "printf" line back into the rombios.c file,
then guests boot without issues.
Very strange!
Tested with only changing the hvmloader, starting with an installed
14344 base, and used the 14345 hvmloader, result is grub failure.
Took a look at the rombios.s code generated in both cases (14344 and
14345) and seems like the "jump to the boot vector" code is not
executed
on the failing case. There is a "default: return" in the function
immediately above an "#if BX_TCGBIOS", and then the non-executed asm
code.
Is bcc not understanding that the asm code following should be executed?
Seems like there is added a default "return" so that the "jump to
boot
vector" code is never executed? appears to be the case..but not sure.
Code snippets are below. Using gcc 3.3.5, but will try 4.x compiler,
although looks like bcc is used to generate the rombios.s file if I read
the Makefile correctly.
Any thoughts appreciated.
[code from rombios.c]
default: return;
}
#if BX_TCGBIOS
tcpa_ipl((Bit32u)bootseg); /* specs: 8.2.3 steps 4 and 5
*/
#endif
/* 14345 removes this: printf("Booting from %x:%x\n", bootseg,
bootip);
*/
/* Jump to the boot vector */
ASM_START
mov bp, sp
;; Build an iret stack frame that will take us to the boot vector.
;; iret pops ip, then cs, then flags, so push them in the opposite
order.
pushf
mov ax, _int18_function.bootseg + 0[bp]
push ax
mov ax, _int18_function.bootip + 0[bp]
push ax
[code from 14344 that boots ]
push -$A[bp]
push -8[bp]
mov bx,#.668
push bx
mov ax,*2
push ax
call _bios_printf
add sp,*8
! 6170
! 6171
! 6172 #asm
!BCC_ASM
_int18_function.bootip set $12
_int18_function.seq_nr set $20
_int18_function.bootchk set $16
_int18_function.bootseg set $14
_int18_function.ebda_seg set $1A
_int18_function.status set $10
_int18_function.bootdrv set $17
_int18_function.bootdev set $18
_int18_function.e set 0
mov bp, sp
;; Build an iret stack frame that will take us to the boot vector.
;; iret pops ip, then cs, then flags, so push them in the opposite
order.
pushf
mov ax, _int18_function.bootseg + 0[bp]
push ax
mov ax, _int18_function.bootip + 0[bp]
[code from 14345 that does not boot ]
mov -$A[bp],ax
! 6161 break;
jmp .654
! 6162
! 6163 default: return;
.667:
mov sp,bp
pop bp
ret
! 6164 }
! 6165
! 6166
! 6167
! 6168
! 6169
! 6170
! 6171
! 6172 #asm
!BCC_ASM
_int18_function.bootip set $12
_int18_function.seq_nr set $20
_int18_function.bootchk set $16
_int18_function.bootseg set $14
_int18_function.ebda_seg set $1A
_int18_function.status set $10
_int18_function.bootdrv set $17
_int18_function.bootdev set $18
_int18_function.e set 0
mov bp, sp
;; Build an iret stack frame that will take us to the boot vector.
;; iret pops ip, then cs, then flags, so push them in the opposite
order.
pushf
mov ax, _int18_function.bootseg + 0[bp]
push ax
mov ax, _int18_function.bootip + 0[bp]
push ax
tom
> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of
> Woller, Thomas
> Sent: Tuesday, March 13, 2007 6:01 PM
> To: xen-devel@lists.xensource.com
> Subject: [Xen-devel] Cs/ 14346 breaks 64bit hv
>
> Hi Keir,
> Appears as if c/s 14346 has broken booting pae and 64bit HVM
> linux guests, on 64bit hv on AMD-V.
>
> Seems like grub can''t find a bootable device, and errors out.
>
> We haven''t tried VT boxes at this point. We just uncovered
> this issue and will investigate more. Looks like 32bit
> hypervisor works ok, but haven''t tested thoroughly.
>
> We''ll look at the patch, but if you have any thoughts.
> Appreciate taking a closer look at the modification.
>
> --Tom
>
> thomas.woller@amd.com
> AMD Corporation
> 5204 E. Ben White Blvd. UBC1
> Austin, Texas 78741
> +1-512-602-0059
>
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
At 01:47 -0500 on 14 Mar (1173836821), Woller, Thomas wrote:> Took a look at the rombios.s code generated in both cases (14344 and > 14345) and seems like the "jump to the boot vector" code is not executed > on the failing case. There is a "default: return" in the function > immediately above an "#if BX_TCGBIOS", and then the non-executed asm > code. > Is bcc not understanding that the asm code following should be executed? > Seems like there is added a default "return" so that the "jump to boot > vector" code is never executed? appears to be the case..but not sure.That would be terrifying. Presumably there''s a jmp past that ret somewhere. I got the impression from your first email that some guests do boot, and that can only happen through that iret. Does adding an explicit "return;" in C after the asm block change the behaviour?> Code snippets are below. Using gcc 3.3.5, but will try 4.x compiler, > although looks like bcc is used to generate the rombios.s file if I read > the Makefile correctly.Yep. What version of bcc are you using? (bcc -v says 0.16.14 for me) Cheers, Tim. -- Tim Deegan <Tim.Deegan@xensource.com>, XenSource UK Limited Registered office c/o EC2Y 5EB, UK; company number 05334508 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
At 09:34 +0000 on 14 Mar (1173864844), Tim Deegan wrote:> Yep. What version of bcc are you using? (bcc -v says 0.16.14 for me)...and can you send me a full copy of rombios.s from the build? I can confirm that 64bit and pae linux guests (though that shouldn''t make a difference) boot fine on my 64bit AMD machine on -unstable tip. Tim. -- Tim Deegan <Tim.Deegan@xensource.com>, XenSource UK Limited Registered office c/o EC2Y 5EB, UK; company number 05334508 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> That would be terrifying. Presumably there''s a jmp past that > ret somewhere. I got the impression from your first email > that some guests do boot, and that can only happen through that iret.I put an int 3 as the first instruction in the ASM block, but didn''t hit it. But maybe int3 handler just does a return. I double checked and 32bit guest on 32bit hv works, but the test was performed with a build from a system with bcc 0.16.17, which is not the version that we are seeing failures from. So, currently failures only occur with the older bcc version.> Yep. What version of bcc are you using? (bcc -v says 0.16.14 for me)Well, I am using SLES9.3 installed base, and bcc -v, says "no input file", a man of bcc doesn''t show any way to get the version :P, hmmm. Here is file, and ls -l info: /usr/bin/bcc: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.2.5, dynamically linked (uses shared libs), stripped -rwxr-xr-x 1 root root 16304 Mar 19 2005 /usr/bin/bcc I''ll find a more recent system to build on with newer bcc. Our build server xen is also based on SLES9.3 with same bcc, and failure occurs from xen builds generated from these also.> > Does adding an explicit "return;" in C after the asm block > change the behaviour?No. didn''t help, code is the same in generated rombios.s. I put in a "call" instead and move the ASM boot code up, but same behavior/code generation. Looks like there has to be some "C" code after the switch or the asm code doesn''t get called.> ...and can you send me a full copy of rombios.s from the build?Attached are 2 rombios.s files in tar.bz2, appended the changeset to the filename. Let me know if you don''t get these, our firewall has been doing some strange things lately. Sure does look like the code generated isn''t correct with older bcc.. But a few more tests will confirm or deny. Thanks for looking at this! Tom _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 14/3/07 15:00, "Woller, Thomas" <thomas.woller@amd.com> wrote:> Sure does look like the code generated isn''t correct with older bcc.. > But a few more tests will confirm or deny.If this is the case I think we should explicitly check for a recent enough bcc during the build, and print a warning (and not build) the rombios if the dev86 tools are too old. We don''t want to have to be worrying about broken toolchains if we can avoid it. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
At 10:00 -0500 on 14 Mar (1173866413), Woller, Thomas wrote:> No. didn''t help, code is the same in generated rombios.s. I put in a > "call" instead and move the ASM boot code up, but same behavior/code > generation. Looks like there has to be some "C" code after the switch > or the asm code doesn''t get called.Yes, it looks like the asm code is being attached to the previous line of C code, which in the 14345 case is the "return" statement inside the switch! My 0.16.14 bcc does the right thing there. Is there a sensible way of getting a version number out of your bcc/as86? Cheers, Tim. -- Tim Deegan <Tim.Deegan@xensource.com>, XenSource UK Limited Registered office c/o EC2Y 5EB, UK; company number 05334508 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> Yes, it looks like the asm code is being attached to the > previous line of C code, which in the 14345 case is the > "return" statement inside the switch! My 0.16.14 bcc does > the right thing there.Yea, I installed bcc 0.16.17 and using this bcc version''s generated hvmloader, allows booting of the guest. And the code in rombios.s looks correct now.> > Is there a sensible way of getting a version number out of > your bcc/as86?I can''t seem to get a version out of bcc, as86, or ld86 that seems to have come from this dev86-0.16-0-187 package installed when I installed SLES9.3. looking around at an objdump isn''t yielding any useful info on first glance. Hmmm. well, we are going to upgrade our buildserver and any personal build boxes. So, I think that we are good with our build process after we upgrade. Not sure what else to do here. Thanks for the lookover, I thought I was misinterpreting the generated code for a bit there. Tom _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel