Xen devel - May 2013 - Xen-unstable 32bit build broken on 64bit systems with multiarch

Hello,

Trying to build 32bit tools on a 64bit multiarch system (Debian Wheezy
in my case) fails.

$ make XEN_TARGET_ARCH=x86_32 -j4
...
/xen-unstable.hg/tools/libxc/../../tools/include  -fPIC -c -o
xc_dom_x86.opic xc_dom_x86.c
gcc  -DPIC -O1 -fno-omit-frame-pointer -m32 -march=i686 -g
-fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes
-Wno-unused-value -Wdeclaration-after-statement
-Wno-unused-but-set-variable  -D__XEN_TOOLS__ -MMD -MF
.xc_cpuid_x86.opic.d  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE
-fno-optimize-sibling-calls -mno-tls-direct-seg-refs -D_GNU_SOURCE
-I../../xen/common/libelf -Werror -Wmissing-prototypes -I.
-I/local/xen-unstable.hg/tools/libxc/../../tools/include  -fPIC -c -o
xc_cpuid_x86.opic xc_cpuid_x86.c
gcc  -DPIC -O1 -fno-omit-frame-pointer -m32 -march=i686 -g
-fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes
-Wno-unused-value -Wdeclaration-after-statement
-Wno-unused-but-set-variable  -D__XEN_TOOLS__ -MMD -MF
.xenctrl_osdep_ENOSYS.opic.d  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE
-fno-optimize-sibling-calls -mno-tls-direct-seg-refs -D_GNU_SOURCE
-I../../xen/common/libelf -Werror -Wmissing-prototypes -I.
-I/local/xen-unstable.hg/tools/libxc/../../tools/include  -fPIC -c -o
xenctrl_osdep_ENOSYS.opic xenctrl_osdep_ENOSYS.c
ar rc libxenctrl.a xc_core.o xc_core_x86.o xc_cpupool.o xc_domain.o
xc_evtchn.o xc_gnttab.o xc_misc.o xc_flask.o xc_physdev.o xc_private.o
xc_sedf.o xc_csched.o xc_csched2.o xc_arinc653.o xc_tbuf.o xc_pm.o
xc_cpu_hotplug.o xc_resume.o xc_tmem.o xc_mem_event.o xc_mem_paging.o
xc_mem_access.o xc_memshr.o xc_hcall_buf.o xc_foreign_memory.o
xtl_core.o xtl_logger_stdio.o xc_pagetab.o xc_linux.o xc_linux_osdep.o
gcc    -Wl,-soname -Wl,libxenctrl.so.4.0 -shared -o libxenctrl.so.4.0.0
xc_core.opic xc_core_x86.opic xc_cpupool.opic xc_domain.opic
xc_evtchn.opic xc_gnttab.opic xc_misc.opic xc_flask.opic xc_physdev.opic
xc_private.opic xc_sedf.opic xc_csched.opic xc_csched2.opic
xc_arinc653.opic xc_tbuf.opic xc_pm.opic xc_cpu_hotplug.opic
xc_resume.opic xc_tmem.opic xc_mem_event.opic xc_mem_paging.opic
xc_mem_access.opic xc_memshr.opic xc_hcall_buf.opic
xc_foreign_memory.opic xtl_core.opic xtl_logger_stdio.opic
xc_pagetab.opic xc_linux.opic xc_linux_osdep.opic -ldl -lpthread
ar rc libxenguest.a xg_private.o xc_suspend.o xc_domain_restore.o
xc_domain_save.o xc_offline_page.o xc_hvm_build.o libelf-tools.o
libelf-loader.o libelf-dominfo.o libelf-relocate.o xc_dom_core.o
xc_dom_boot.o xc_dom_elfloader.o xc_dom_bzimageloader.o
xc_dom_binloader.o xc_dom_compat_linux.o xc_dom_x86.o xc_cpuid_x86.o
/usr/bin/ld: i386 architecture of input file `xc_core.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_core_x86.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_cpupool.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_domain.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_evtchn.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_gnttab.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_misc.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_flask.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_physdev.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_private.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_sedf.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_csched.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_csched2.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_arinc653.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_tbuf.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_pm.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_cpu_hotplug.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_resume.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_tmem.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_mem_event.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_mem_paging.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_mem_access.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_memshr.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_hcall_buf.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_foreign_memory.opic''
is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xtl_core.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xtl_logger_stdio.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_pagetab.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_linux.opic'' is
incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `xc_linux_osdep.opic'' is
incompatible with i386:x86-64 output
collect2: error: ld returned 1 exit status
make[3]: *** [libxenctrl.so.4.0.0] Error 1
make[3]: *** Waiting for unfinished jobs....
make[3]: Leaving directory `/local/xen-unstable.hg/tools/libxc''


I suspect that the issue is the lack of an -m32 for

gcc    -Wl,-soname -Wl,libxenctrl.so.4.0 -shared -o libxenctrl.so.4.0.0
xc_core.opic xc_core_x86.opic xc_cpupool.opic xc_domain.opic
xc_evtchn.opic xc_gnttab.opic xc_misc.opic xc_flask.opic xc_physdev.opic
xc_private.opic xc_sedf.opic xc_csched.opic xc_csched2.opic
xc_arinc653.opic xc_tbuf.opic xc_pm.opic xc_cpu_hotplug.opic
xc_resume.opic xc_tmem.opic xc_mem_event.opic xc_mem_paging.opic
xc_mem_access.opic xc_memshr.opic xc_hcall_buf.opic
xc_foreign_memory.opic xtl_core.opic xtl_logger_stdio.opic
xc_pagetab.opic xc_linux.opic xc_linux_osdep.opic -ldl -lpthread

I have tried a few ideas but not managed to get a successful build. 
Does anyone with more knowledge in this area have any suggestions?

~Andrew

On Thu, 2013-05-23 at 10:39 +0100, Andrew Cooper wrote:
> Hello,
> 
> Trying to build 32bit tools on a 64bit multiarch system (Debian Wheezy
> in my case) fails.
I''m not sure that multiarch is actually quite ready for cross building
in the "biarch" -m32 manner, I think til now they have mostly
concentrated on being able to cross install runtime packages and proper
cross building (i.e. with $triplet-gcc not gcc -m32).

This is all based on vague observations on e.g. Debian devel and not
something I''m 100% sure on.
> $ make XEN_TARGET_ARCH=x86_32 -j4
[...]
> I suspect that the issue is the lack of an -m32 for
> 
> gcc    -Wl,-soname -Wl,libxenctrl.so.4.0 -shared -o libxenctrl.so.4.0.0
> xc_core.opic xc_core_x86.opic xc_cpupool.opic xc_domain.opic
> xc_evtchn.opic xc_gnttab.opic xc_misc.opic xc_flask.opic xc_physdev.opic
> xc_private.opic xc_sedf.opic xc_csched.opic xc_csched2.opic
> xc_arinc653.opic xc_tbuf.opic xc_pm.opic xc_cpu_hotplug.opic
> xc_resume.opic xc_tmem.opic xc_mem_event.opic xc_mem_paging.opic
> xc_mem_access.opic xc_memshr.opic xc_hcall_buf.opic
> xc_foreign_memory.opic xtl_core.opic xtl_logger_stdio.opic
> xc_pagetab.opic xc_linux.opic xc_linux_osdep.opic -ldl -lpthread
> 
> I have tried a few ideas but not managed to get a successful build. 
> Does anyone with more knowledge in this area have any suggestions?
What happens if you add the -m32? Do the error messages differ?

Ian.

On 23/05/13 11:01, Ian Campbell wrote:
> On Thu, 2013-05-23 at 10:39 +0100, Andrew Cooper wrote:
>> Hello,
>>
>> Trying to build 32bit tools on a 64bit multiarch system (Debian Wheezy
>> in my case) fails.
> I''m not sure that multiarch is actually quite ready for cross
building
> in the "biarch" -m32 manner, I think til now they have mostly
> concentrated on being able to cross install runtime packages and proper
> cross building (i.e. with $triplet-gcc not gcc -m32).
>
> This is all based on vague observations on e.g. Debian devel and not
> something I''m 100% sure on.
>
>> $ make XEN_TARGET_ARCH=x86_32 -j4
> [...]
>> I suspect that the issue is the lack of an -m32 for
>>
>> gcc    -Wl,-soname -Wl,libxenctrl.so.4.0 -shared -o libxenctrl.so.4.0.0
>> xc_core.opic xc_core_x86.opic xc_cpupool.opic xc_domain.opic
>> xc_evtchn.opic xc_gnttab.opic xc_misc.opic xc_flask.opic
xc_physdev.opic
>> xc_private.opic xc_sedf.opic xc_csched.opic xc_csched2.opic
>> xc_arinc653.opic xc_tbuf.opic xc_pm.opic xc_cpu_hotplug.opic
>> xc_resume.opic xc_tmem.opic xc_mem_event.opic xc_mem_paging.opic
>> xc_mem_access.opic xc_memshr.opic xc_hcall_buf.opic
>> xc_foreign_memory.opic xtl_core.opic xtl_logger_stdio.opic
>> xc_pagetab.opic xc_linux.opic xc_linux_osdep.opic -ldl -lpthread
>>
>> I have tried a few ideas but not managed to get a successful build. 
>> Does anyone with more knowledge in this area have any suggestions?
> What happens if you add the -m32? Do the error messages differ?
>
> Ian.
>
With -m32, I fail to link against -lbz2 and -lz.  I thought I had
installed both variants of these libraries but I am now not so sure.

~Andrew

On Thu, 2013-05-23 at 11:10 +0100, Andrew Cooper wrote:
> With -m32, I fail to link against -lbz2 and -lz.  I thought I had
> installed both variants of these libraries but I am now not so sure.
Although perhaps core libraries like libz have already been done I think
you can''t in general currently have two different arch''s -dev
package
installed simultaneously. AIUI that''s Phase II of multiarch (i.e.
splitting /usr/include and *.so into multiarch paths). This effectively
limits you to either a 32 or 64 bit (or cross) build environment even
when you runtime environment is entirely multiarch.

Ian.

On 23/05/13 11:23, Ian Campbell wrote:
> On Thu, 2013-05-23 at 11:10 +0100, Andrew Cooper wrote:
>> With -m32, I fail to link against -lbz2 and -lz.  I thought I had
>> installed both variants of these libraries but I am now not so sure.
> Although perhaps core libraries like libz have already been done I think
> you can''t in general currently have two different arch''s
-dev package
> installed simultaneously. AIUI that''s Phase II of multiarch (i.e.
> splitting /usr/include and *.so into multiarch paths). This effectively
> limits you to either a 32 or 64 bit (or cross) build environment even
> when you runtime environment is entirely multiarch.
>
> Ian.
>
Ok - lets settle with "Dont Do That Then" for now.

~Andrew