xen discuss - Oct 2008 - 32bit PV domU on 64bit snv99/xVM

Hi,

I am trying to virt-install Fedora 8 i386 (32bit) as a domU on a snv99/xVM 64bit
system and, although the installation procedure itself works smoothly, upon the
first boot I am constantly getting the following error:

    libvir: Xen Daemon error : POST operation failed: (xend.err "(2,
''Invalid kernel'', ''xc_dom_find_loader: no loader
found\\n'')")
    error: Failed to start domain YYY

The GRUB config of the installed domU seems to be OK, that is, it refers to the
vmlinuz-2.6.21-2950.fc8xen kernel. I am pretty sure the kernel is PAE-enabled:

    # grep "PAE\|HIGHMEM" /g.boot/config-2.6.21-2950.fc8xen
    # CONFIG_NOHIGHMEM is not set
    # CONFIG_HIGHMEM4G is not set
    CONFIG_HIGHMEM64G=y
    CONFIG_HIGHMEM=y
    CONFIG_X86_PAE=y
    CONFIG_DEBUG_HIGHMEM=y

Is this supposed to work in build 99, which, as far as I understand, includes
some modification of Xen 3.1.4? According to
http://wiki.xensource.com/xenwiki/XenFaq#head-5f7176b3909cb0382cece43a6a8fc25a3a114e93,
32bit PV DomUs under 64bit hypervisor are supported starting from Xen 3.1.

And, I have an equivalent setup working fine with 64bit Fedora 8.

Thank you,
Vasiliy
--
This message posted from opensolaris.org

On Thu, Oct 16, 2008 at 06:52:51AM -0700, Vasiliy Baranov wrote:
> I am trying to virt-install Fedora 8 i386 (32bit) as a domU on a snv99/xVM
64bit system and, although the installation procedure itself works smoothly,
upon the first boot I am constantly getting the following error:
32 on 64 PV is not a supported configuration. Boot into 32-bit, or just
use a 64-bit Fedora ISO.

Please file a bug about the failure though.

regards
john

so reemphasizing that this is a 32PAE kernel, are you saying that it a 
bug that xm info is reporting xen_caps : xen-3.0-x86_32p?

=Ron
John Levon wrote:
> On Thu, Oct 16, 2008 at 06:52:51AM -0700, Vasiliy Baranov wrote:
>
>   
>> I am trying to virt-install Fedora 8 i386 (32bit) as a domU on a
snv99/xVM 64bit system and, although the installation procedure itself works
smoothly, upon the first boot I am constantly getting the following error:
>>     
>
> 32 on 64 PV is not a supported configuration. Boot into 32-bit, or just
> use a 64-bit Fedora ISO.
>
> Please file a bug about the failure though.
>
> regards
> john
> _______________________________________________
> xen-discuss mailing list
> xen-discuss@opensolaris.org
>

On Thu, Oct 16, 2008 at 01:53:03PM -0400, Ronald J Mann wrote:
> so reemphasizing that this is a 32PAE kernel, are you saying that it a 
> bug that xm info is reporting xen_caps : xen-3.0-x86_32p?
No. There''s a difference between what sometimes works, what might work,
what does work, and what works well enough to be known and supported.

regards
john

It works at Xen 3.2.1 F8 Dom0 (x86_64).
http://librenix.com/?inode=12222
--
This message posted from opensolaris.org

FWIW, I got it kind of working now.

There was one suspicious thing about this failure. While in the 64 bit 
Fedora 8 case the domain builder was considering the guest kernel a 
gzipped ELF image and was able to use it successfully, in the 32 bit 
case it was not seeing the guest kernel as either gzipped or 
uncompressed executable. Namely, when a 64 bit guest was starting, the 
domain builder was logging the following (in 
/var/log/xen/domain-builder-ng.log):

   xc_dom_allocate: cmdline="ro root=/dev/VolGroup00/LogVol00 
console=xvc0 rhgb quiet", features=""
   xc_dom_kernel_file:
filename="/var/run/xend/boot/boot_kernel.BtN5gl"
   xc_dom_malloc_filemap    : 1915 kB
   xc_dom_malloc            : 8991 kB
   xc_dom_do_gunzip: unzip ok, 0x1ded4d -> 0x8c7c08
   xc_dom_ramdisk_file:
filename="/var/run/xend/boot/boot_ramdisk.FTyUVc"
   xc_dom_malloc_filemap    : 3714 kB
   xc_dom_boot_xen_init: ver 3.1, caps xen-3.0-x86_64 xen-3.0-x86_32p 
hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
   xc_dom_parse_image: called
   xc_dom_find_loader: trying ELF-generic loader ... OK
   elf_parse_binary: phdr: paddr=0xffffffff80200000 memsz=0x33a610
   ...
   xc_dom_boot_image: called
   arch_setup_bootearly: doing nothing
   xc_dom_compat_check: supported guest type: xen-3.0-x86_64 <= matches
   xc_dom_compat_check: supported guest type: xen-3.0-x86_32p
   xc_dom_compat_check: supported guest type: hvm-3.0-x86_32
   xc_dom_compat_check: supported guest type: hvm-3.0-x86_32p
   xc_dom_compat_check: supported guest type: hvm-3.0-x86_64
   ...

while in the 32 bit case the domain builder output was:

   xc_dom_allocate: cmdline="ro root=/dev/VolGroup00/LogVol00 
console=xvc0 rhgb quiet", features=""
   xc_dom_kernel_file:
filename="/var/run/xend/boot/boot_kernel.YO8N4O"
   xc_dom_malloc_filemap    : 2175 kB
   xc_dom_ramdisk_file:
filename="/var/run/xend/boot/boot_ramdisk.g3Xb9t"
   xc_dom_malloc_filemap    : 3461 kB
   xc_dom_boot_xen_init: ver 3.1, caps xen-3.0-x86_64 xen-3.0-x86_32p 
hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
   xc_dom_parse_image: called
   xc_dom_find_loader: trying ELF-generic loader ... failed
   xc_dom_find_loader: trying multiboot-binary loader ... failed
   xc_dom_core.c:495: panic: xc_dom_find_loader: no loader found
   xc_dom_release: called

Note that in the 32 bit case the domain builder was not even trying to 
gunzip the kernel.

Investigating this turned into a quite an adventure, but to be short, at 
some moment I was able to boot the installed 32 bit Fedora 8 guest in 
the so-called "rescue" mode (by using virt-install and passing the 
''rescue'' argument to the anaconda installer) and install a
newer kernel
RPM (kernel-xen-2.6.21.7-5.fc8.i686.rpm) directly into the guest. I was 
then able to confirm that the updated kernel was a gzip archive. 
However, after reboot I was still having the same problem and seeing the 
same messages in the logs. I then booted the guest in the rescue mode 
again and found that the kernel was no longer a gzip archive, 
surprisingly. Then I re-installed the kernel, copied it to dom0, and 
changed the guest XML config to use the copied kernel instead of pygrub. 
This way it worked. The 32 bit PV guest was able to boot successfully, 
and the domain builder output was as:

   xc_dom_allocate: cmdline=" ro root=/dev/VolGroup00/LogVol00 
console=xvc0 rhgb quiet ", features=""
   xc_dom_kernel_file:
filename="/files/kernels/vmlinuz-2.6.21.7-5.fc8xen"
   xc_dom_malloc_filemap    : 2175 kB
   xc_dom_malloc            : 4842 kB
   xc_dom_do_gunzip: unzip ok, 0x21fc43 -> 0x4bab44
   xc_dom_ramdisk_file: 
filename="/files/kernels/initrd-2.6.21.7-5.fc8xen.img"
   xc_dom_malloc_filemap    : 3461 kB
   xc_dom_boot_xen_init: ver 3.1, caps xen-3.0-x86_64 xen-3.0-x86_32p 
hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
   xc_dom_parse_image: called
   xc_dom_find_loader: trying ELF-generic loader ... OK
   elf_parse_binary: phdr: paddr=0xc1000000 memsz=0x2d347c
   ...
   xc_dom_boot_image: called
   arch_setup_bootearly: doing nothing
   xc_dom_compat_check: supported guest type: xen-3.0-x86_64
   xc_dom_compat_check: supported guest type: xen-3.0-x86_32p <= matches
   xc_dom_compat_check: supported guest type: hvm-3.0-x86_32
   xc_dom_compat_check: supported guest type: hvm-3.0-x86_32p
   xc_dom_compat_check: supported guest type: hvm-3.0-x86_64
   ...

I then compared the kernel copied to dom0 with the kernel found in domU. 
The domU kernel was different in that its first 1024 bytes were complete 
garbage, everything else including the file size was the same.

I thought this corruption problem could be part of why "32 on 64 PV is 
not a supported configuration", but, per you suggestion, I repeated the 
entire procedure while booted into 32 bit xVM and had exactly the same 
problem. I was also able to create a 32 bit PV PAE openSUSE 10.3 guest 
on 64 bit xVM and have not seen any similar problems so far.

Any thoughts?

Thank you,
Vasiliy

On Thu, Oct 23, 2008 at 04:02:28PM +0400, Vasiliy Baranov wrote:
> I thought this corruption problem could be part of why "32 on 64 PV is
> not a supported configuration", but, per you suggestion, I repeated
the
> entire procedure while booted into 32 bit xVM and had exactly the same 
Can you clarify: if you boot into 32-bit xVM, and use virt-install, a
32-bit Fedora 8 will not install?

If so, that''s a bug, and please file it, solaris/xvm/utility

regards
john

John Levon wrote:
> On Thu, Oct 23, 2008 at 04:02:28PM +0400, Vasiliy Baranov wrote:
>
>   
>> I thought this corruption problem could be part of why "32 on 64
PV is
>> not a supported configuration", but, per you suggestion, I
repeated the
>> entire procedure while booted into 32 bit xVM and had exactly the same 
>>     
>
> Can you clarify: if you boot into 32-bit xVM, and use virt-install, a
> 32-bit Fedora 8 will not install?
>   
Correct. Not only will it not start but the domU kernel will get 
corrupted somehow. I''ll file a bug.

Thank you,
Vasiliy
> If so, that''s a bug, and please file it, solaris/xvm/utility
>
> regards
> john
>