Hi All, I have two Xen machines, both running 4.12 compiled from source. One is running SL6.2 with a kernel compiled from vanilla source (3.2.0) and the second is using the stock kernel from Ubuntu (3.2.0-23-generic). I moved a pv CentOS 5.8 vm from the SL machine to the Ubuntu machine and noticed a difference when starting (i.e. an error) "xl -v create" gives... SL 6.2 domainbuilder: detail: xc_dom_allocate: cmdline="ro root=LABEL=/ console=xvc0 ", features="(null)" domainbuilder: detail: xc_dom_kernel_mem: called domainbuilder: detail: xc_dom_malloc : 10836 kB domainbuilder: detail: xc_dom_do_gunzip: unzip ok, 0x21bbf4 -> 0xa952d8 domainbuilder: detail: xc_dom_ramdisk_mem: called domainbuilder: detail: xc_dom_boot_xen_init: ver 4.1, caps xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 domainbuilder: detail: xc_dom_parse_image: called domainbuilder: detail: xc_dom_find_loader: trying ELF-generic loader ... domainbuilder: detail: loader probe OK Ubuntu 12.04 domainbuilder: detail: xc_dom_allocate: cmdline="ro root=LABEL=/ console=xvc0 ", features="(null)" domainbuilder: detail: xc_dom_kernel_mem: called domainbuilder: detail: xc_dom_malloc : 10836 kB domainbuilder: detail: xc_dom_do_gunzip: unzip ok, 0x21bbf4 -> 0xa952d8 domainbuilder: detail: xc_dom_ramdisk_mem: called domainbuilder: detail: xc_dom_boot_xen_init: ver 4.1, caps xen-3.0-x86_64 xen-3.0-x86_32p domainbuilder: detail: xc_dom_parse_image: called domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader ... domainbuilder: detail: loader probe failed domainbuilder: detail: xc_dom_find_loader: trying Linux bzImage loader ... xc: error: panic: xc_dom_bzimageloader.c:556: xc_dom_probe_bzimage_kernel: kernel is not a bzImage: Invalid kernel domainbuilder: detail: loader probe failed domainbuilder: detail: xc_dom_find_loader: trying ELF-generic loader ... domainbuilder: detail: loader probe OK Both boot fine, and it looks like the Ubuntu version is trying to do something extra. I googled the error message and I am not concerned about it as it appears to be a spurious error caused by an extra probe. Apart from the SL vs Ubuntu difference, I thought I had the machines pretty similar from a Xen perspective. Can anyone please explain why these extra probes are talking place and what in my (possibly build) environment might have caused this to happen (or indeed cause this *not* to happen on the SL box). Thanks in advance, Ian. xl info for Ubuntu: host : xen4 release : 3.2.0-23-generic version : #36-Ubuntu SMP Tue Apr 10 20:39:51 UTC 2012 machine : x86_64 nr_cpus : 4 nr_nodes : 1 cores_per_socket : 2 threads_per_core : 2 cpu_mhz : 1800 hw_caps : bfebfbff:20100800:00000000:00000940:0040e31d:00000000:00000001:00000000 virt_caps : total_memory : 4078 free_memory : 1155 free_cpus : 0 xen_major : 4 xen_minor : 1 xen_extra : .2 xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p xen_scheduler : credit xen_pagesize : 4096 platform_params : virt_start=0xffff800000000000 xen_changeset : unavailable xen_commandline : placeholder cc_compiler : gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) cc_compile_by : root cc_compile_domain : cc_compile_date : Mon May 21 00:50:17 BST 2012 xend_config_format : 4 xl info for SL: host : xen3 release : 3.2.0+ version : #3 SMP Sat Jan 21 12:27:48 GMT 2012 machine : x86_64 nr_cpus : 4 nr_nodes : 1 cores_per_socket : 4 threads_per_core : 1 cpu_mhz : 2300 hw_caps : 178bf3ff:efd3fbff:00000000:00001310:00802001:00000000:000007ff:00000000 virt_caps : hvm total_memory : 8159 free_memory : 4878 free_cpus : 0 xen_major : 4 xen_minor : 1 xen_extra : .2 xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 xen_scheduler : credit xen_pagesize : 4096 platform_params : virt_start=0xffff800000000000 xen_changeset : unavailable xen_commandline : loglvl=all guesloglvl=all cc_compiler : gcc version 4.4.5 20110214 (Red Hat 4.4.5-6) (GCC) cc_compile_by : root cc_compile_domain : (none) cc_compile_date : Sun Jan 22 01:18:42 GMT 2012 xend_config_format : 4
On Wed, 2012-05-30 at 01:15 +0100, Ian wrote:> Hi All, > > I have two Xen machines, both running 4.12 compiled from source. One is > running SL6.2 with a kernel compiled from vanilla source (3.2.0) and the > second is using the stock kernel from Ubuntu (3.2.0-23-generic). > > I moved a pv CentOS 5.8 vm from the SL machine to the Ubuntu machine and > noticed a difference when starting (i.e. an error) > > "xl -v create" gives... > > > SL 6.2 > > domainbuilder: detail: xc_dom_allocate: cmdline="ro root=LABEL=/ > console=xvc0 ", features="(null)" > domainbuilder: detail: xc_dom_kernel_mem: called > domainbuilder: detail: xc_dom_malloc : 10836 kB > domainbuilder: detail: xc_dom_do_gunzip: unzip ok, 0x21bbf4 -> 0xa952d8 > domainbuilder: detail: xc_dom_ramdisk_mem: called > domainbuilder: detail: xc_dom_boot_xen_init: ver 4.1, caps > xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p > hvm-3.0-x86_64 > domainbuilder: detail: xc_dom_parse_image: called > domainbuilder: detail: xc_dom_find_loader: trying ELF-generic loader ... > domainbuilder: detail: loader probe OK > > Ubuntu 12.04 > > domainbuilder: detail: xc_dom_allocate: cmdline="ro root=LABEL=/ > console=xvc0 ", features="(null)" > domainbuilder: detail: xc_dom_kernel_mem: called > domainbuilder: detail: xc_dom_malloc : 10836 kB > domainbuilder: detail: xc_dom_do_gunzip: unzip ok, 0x21bbf4 -> 0xa952d8 > domainbuilder: detail: xc_dom_ramdisk_mem: called > domainbuilder: detail: xc_dom_boot_xen_init: ver 4.1, caps > xen-3.0-x86_64 xen-3.0-x86_32p > domainbuilder: detail: xc_dom_parse_image: called > domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary > loader ... > domainbuilder: detail: loader probe failed > domainbuilder: detail: xc_dom_find_loader: trying Linux bzImage loader ... > xc: error: panic: xc_dom_bzimageloader.c:556: > xc_dom_probe_bzimage_kernel: kernel is not a bzImage: Invalid kernel > domainbuilder: detail: loader probe failed > domainbuilder: detail: xc_dom_find_loader: trying ELF-generic loader ... > domainbuilder: detail: loader probe OK > > > Both boot fine, and it looks like the Ubuntu version is trying to do > something extra. I googled the error message and I am not concerned > about it as it appears to be a spurious error caused by an extra probe.That''s right, it''s spurious. I have an outstanding patch on xen-devel@ to tone it down, it shouldn''t be a panic!> Apart from the SL vs Ubuntu difference, I thought I had the machines > pretty similar from a Xen perspective. Can anyone please explain why > these extra probes are talking place and what in my (possibly build) > environment might have caused this to happen (or indeed cause this *not* > to happen on the SL box).If the kernels being used are identical then I don''t know. Are you booting the kernel from inside the CentOS VM (e.g. via pygrub) or are you picking it up from the dom0 filesystem? If the kernels are not identical then the message is probably because one kernel is a bzImage (often this == pvops) and the other is a compressed ELF file (often this == classic XenoLinux). You seem to be building some kernels yourself but FWIW the out of the box kernel on Ubuntu is pvops while SL and CentOS 5 are both classic. Ian.
Thanks for the reply.> >If the kernels being used are identical then I don''t know. Are you >booting the kernel from inside the CentOS VM (e.g. via pygrub) or are >you picking it up from the dom0 filesystem? > >If the kernels are not identical then the message is probably because >one kernel is a bzImage (often this == pvops) and the other is a >compressed ELF file (often this == classic XenoLinux).I knew I''d miss some info. They are both the same virtual machine, identical create file (bar slight change in LV path), identical LV ( dd''d with the other machine) and using pygrub.> >You seem to be building some kernels yourself but FWIW the out of the >box kernel on Ubuntu is pvops while SL and CentOS 5 are both classic. >The only kernel I built myself was the SL Dom0 (as it was SL 6.2). Kernel in VM is 2.6.18-308.4.1.el5xen Here is my config file.... name = "mail" memory = "1524" disk = [ ''phy:/dev/xen4/mail-root,xvda,w'',''phy:/dev/xen4/mail-swap,xvdb,w''] vif = [ ''mac=00:16:31:50:01:02, bridge=eth0'' ] bootloader="/usr/bin/pygrub" vcpus=2 cpus=''1-3'' on_reboot = ''restart'' on_crash = ''restart'' Nothing too startling there. Thanks again, Ian.
On 30/05/12 06:44, Ian Campbell wrote:> On Wed, 2012-05-30 at 01:15 +0100, Ian wrote: >> Hi All, >> >> I have two Xen machines, both running 4.12 compiled from source. One is >> running SL6.2 with a kernel compiled from vanilla source (3.2.0) and the >> second is using the stock kernel from Ubuntu (3.2.0-23-generic). >> >> I moved a pv CentOS 5.8 vm from the SL machine to the Ubuntu machine and >> noticed a difference when starting (i.e. an error) >> >> "xl -v create" gives... >> >> >> SL 6.2 >> >> domainbuilder: detail: xc_dom_allocate: cmdline="ro root=LABEL=/ >> console=xvc0 ", features="(null)" >> domainbuilder: detail: xc_dom_kernel_mem: called >> domainbuilder: detail: xc_dom_malloc : 10836 kB >> domainbuilder: detail: xc_dom_do_gunzip: unzip ok, 0x21bbf4 -> 0xa952d8 >> domainbuilder: detail: xc_dom_ramdisk_mem: called >> domainbuilder: detail: xc_dom_boot_xen_init: ver 4.1, caps >> xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p >> hvm-3.0-x86_64 >> domainbuilder: detail: xc_dom_parse_image: called >> domainbuilder: detail: xc_dom_find_loader: trying ELF-generic loader ... >> domainbuilder: detail: loader probe OK >> >> Ubuntu 12.04 >> >> domainbuilder: detail: xc_dom_allocate: cmdline="ro root=LABEL=/ >> console=xvc0 ", features="(null)" >> domainbuilder: detail: xc_dom_kernel_mem: called >> domainbuilder: detail: xc_dom_malloc : 10836 kB >> domainbuilder: detail: xc_dom_do_gunzip: unzip ok, 0x21bbf4 -> 0xa952d8 >> domainbuilder: detail: xc_dom_ramdisk_mem: called >> domainbuilder: detail: xc_dom_boot_xen_init: ver 4.1, caps >> xen-3.0-x86_64 xen-3.0-x86_32p >> domainbuilder: detail: xc_dom_parse_image: called >> domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary >> loader ... >> domainbuilder: detail: loader probe failed >> domainbuilder: detail: xc_dom_find_loader: trying Linux bzImage loader ... >> xc: error: panic: xc_dom_bzimageloader.c:556: >> xc_dom_probe_bzimage_kernel: kernel is not a bzImage: Invalid kernel >> domainbuilder: detail: loader probe failed >> domainbuilder: detail: xc_dom_find_loader: trying ELF-generic loader ... >> domainbuilder: detail: loader probe OK >> >> >> Both boot fine, and it looks like the Ubuntu version is trying to do >> something extra. I googled the error message and I am not concerned >> about it as it appears to be a spurious error caused by an extra probe. >After further digging, it would seem that the SL machine was performing the same actions. Further digging in the source code suggests that the loaders are "pluggable" and are "registered". Fair to say that for some reason, they are registered in a different order on the different machines. Most of this stretches my limited understanding of C and I am not figuring out how the ordering of the registration is done. Maybe the compiler, maybe make. I am starting to think this isn''t worth worrying about.
On Thu, 2012-05-31 at 02:32 +0100, Ian wrote:> On 30/05/12 06:44, Ian Campbell wrote: > > On Wed, 2012-05-30 at 01:15 +0100, Ian wrote: > >> Hi All, > >> > >> I have two Xen machines, both running 4.12 compiled from source. One is > >> running SL6.2 with a kernel compiled from vanilla source (3.2.0) and the > >> second is using the stock kernel from Ubuntu (3.2.0-23-generic). > >> > >> I moved a pv CentOS 5.8 vm from the SL machine to the Ubuntu machine and > >> noticed a difference when starting (i.e. an error) > >> > >> "xl -v create" gives... > >> > >> > >> SL 6.2 > >> > >> domainbuilder: detail: xc_dom_allocate: cmdline="ro root=LABEL=/ > >> console=xvc0 ", features="(null)" > >> domainbuilder: detail: xc_dom_kernel_mem: called > >> domainbuilder: detail: xc_dom_malloc : 10836 kB > >> domainbuilder: detail: xc_dom_do_gunzip: unzip ok, 0x21bbf4 -> 0xa952d8 > >> domainbuilder: detail: xc_dom_ramdisk_mem: called > >> domainbuilder: detail: xc_dom_boot_xen_init: ver 4.1, caps > >> xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p > >> hvm-3.0-x86_64 > >> domainbuilder: detail: xc_dom_parse_image: called > >> domainbuilder: detail: xc_dom_find_loader: trying ELF-generic loader ... > >> domainbuilder: detail: loader probe OK > >> > >> Ubuntu 12.04 > >> > >> domainbuilder: detail: xc_dom_allocate: cmdline="ro root=LABEL=/ > >> console=xvc0 ", features="(null)" > >> domainbuilder: detail: xc_dom_kernel_mem: called > >> domainbuilder: detail: xc_dom_malloc : 10836 kB > >> domainbuilder: detail: xc_dom_do_gunzip: unzip ok, 0x21bbf4 -> 0xa952d8 > >> domainbuilder: detail: xc_dom_ramdisk_mem: called > >> domainbuilder: detail: xc_dom_boot_xen_init: ver 4.1, caps > >> xen-3.0-x86_64 xen-3.0-x86_32p > >> domainbuilder: detail: xc_dom_parse_image: called > >> domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary > >> loader ... > >> domainbuilder: detail: loader probe failed > >> domainbuilder: detail: xc_dom_find_loader: trying Linux bzImage loader ... > >> xc: error: panic: xc_dom_bzimageloader.c:556: > >> xc_dom_probe_bzimage_kernel: kernel is not a bzImage: Invalid kernel > >> domainbuilder: detail: loader probe failed > >> domainbuilder: detail: xc_dom_find_loader: trying ELF-generic loader ... > >> domainbuilder: detail: loader probe OK > >> > >> > >> Both boot fine, and it looks like the Ubuntu version is trying to do > >> something extra. I googled the error message and I am not concerned > >> about it as it appears to be a spurious error caused by an extra probe. > > > After further digging, it would seem that the SL machine was performing > the same actions. Further digging in the source code suggests that the > loaders are "pluggable" and are "registered". Fair to say that for some > reason, they are registered in a different order on the different > machines. Most of this stretches my limited understanding of C and I am > not figuring out how the ordering of the registration is done. Maybe the > compiler, maybe make.Right.> I am starting to think this isn''t worth worrying about.I agree, it''s not. Ian.