Hi there, I've had issue with the integration of syslinux into the buildroot project [0]. After some research, it turned out that we were wrongly using the host toolchain instead of the target toolchain to build syslinux. This gave us bigger issues (for instance) when trying to link syslinux with gnu-efi, that was built using the target toolchain. The first one forces ld to output i386 binary when processing the binary files generated by nasm (for consistency with the other .o files), and the second one explicitely uses the host toolchain to build the utilities under utils/. Those two patches turned out to be necessary in order to have a proper build. please consider them for upstream inclusion. There was an aditionnal third patch [1] I required, but that one isn't authored by me. I would be grateful if you integrate it in your next release as well. Best Regards, Ben. [0] https://buildroot.org [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?att=1;bug=846679;filename=syslinux_6.03%2Bdfsg-14.1.debdiff;msg=10 Beno?t Allard (2): memdisk: Force ld output format to 32-bits utils: Use the host toolchain to build. memdisk/Makefile | 2 +- utils/Makefile | 5 +++-- 2 files changed, 4 insertions(+), 3 deletions(-) -- 2.1.4
Benoît Allard
2017-Apr-05 14:32 UTC
[syslinux] [PATCH 1/2] memdisk: Force ld output format to 32-bits
On toolchains where the default output is x86_64, we need to be consistent with the other .o files Signed-off-by: Beno?t Allard <benoit.allard at greenbone.net> --- memdisk/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/memdisk/Makefile b/memdisk/Makefile index 42e56e0..ccd5738 100644 --- a/memdisk/Makefile +++ b/memdisk/Makefile @@ -78,7 +78,7 @@ memdisk16.o: memdisk16.asm $(NASM) -f bin $(NASMOPT) $(NFLAGS) $(NINCLUDE) -o $@ -l $*.lst $< memdisk_%.o: memdisk_%.bin - $(LD) -r -b binary -o $@ $< + $(LD) --oformat elf32-i386 -r -b binary -o $@ $< memdisk16.elf: $(OBJS16) $(LD) -Ttext 0 -o $@ $^ -- 2.1.4
Benoît Allard
2017-Apr-05 14:32 UTC
[syslinux] [PATCH 2/2] utils: Use the host toolchain to build.
The utilities are meant to run on the host machine, hence must be built using the host toolchain. Signed-off-by: Beno?t Allard <benoit.allard at greenbone.net> --- utils/Makefile | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/utils/Makefile b/utils/Makefile index dfe6259..f31f117 100644 --- a/utils/Makefile +++ b/utils/Makefile @@ -17,8 +17,9 @@ VPATH = $(SRC) include $(MAKEDIR)/syslinux.mk -CFLAGS = $(GCCWARN) -Os -fomit-frame-pointer -D_FILE_OFFSET_BITS=64 -I$(SRC) -LDFLAGS = -O2 +CC = $(CC_FOR_BUILD) +CFLAGS = $(CFLAGS_FOR_BUILD) $(GCCWARN) -Os -fomit-frame-pointer -D_FILE_OFFSET_BITS=64 -I$(SRC) +LDFLAGS = $(LDFLAGS_FOR_BUILD) -O2 C_TARGETS = isohybrid gethostip memdiskfind SCRIPT_TARGETS = mkdiskimage -- 2.1.4
> Hi there, > > I've had issue with the integration of syslinux into the buildroot project > [0]. After some research, it turned out that we were wrongly using the host > toolchain instead of the target toolchain to build syslinux. This gave us > bigger issues (for instance) when trying to link syslinux with gnu-efi, that > was built using the target toolchain. > > The first one forces ld to output i386 binary when processing the binary > files generated by nasm (for consistency with the other .o files), and the > second one explicitely uses the host toolchain to build the utilities under > utils/. > > Those two patches turned out to be necessary in order to have a proper build. > please consider them for upstream inclusion. > > There was an aditionnal third patch [1] I required, but that one isn't > authored by me. I would be grateful if you integrate it in your next release > as well. > > Best Regards, > Ben. >I'm not not sure the assumptions are always true (for every downstream use case / project / distro). Expressions such as "The utilities are meant to run on the host machine, hence must be built using the host toolchain." might be relevant for certain use cases whereas not necessarily true for others. What would happen if a user, while troubleshooting, wants to use a binary - just for the sake of simplicity, say we are talking about memdisk and/or the isohybrid tool - that was created on a different host than this user has/uses? We should want to be able to tell such user things like "try the pre-built official binaries included in the official distribution archives located at kernel.org". In a similar (but not the same) line of thinking , there are Linux distributions that are no longer building packages for xi86, and so they don't build EFI binaries for UEFI IA32. I am sure that a PC with a UEFI IA32 firmware is not from the same era than a "686" PC, and they do not use such a similar microprocessor. UEFI IA32 binaries should still be built, but such distros don't do it anymore; IMHO, they still should. We should be aware about the broad consequences (for others) when we assume that "my use-case is _the_ (only/best) case. Regards, Ady.
Replying to my own mail as I subscribe to the ML after the answer came, and thus never received it in my mailbox. On Wed, 5 Apr 2017 16:32:04 +0200 Beno?t Allard <benoit.allard at greenbone.net> wrote:> Hi there, > > I've had issue with the integration of syslinux into the buildroot > project [0]. After some research, it turned out that we were wrongly > using the host toolchain instead of the target toolchain to build > syslinux. This gave us bigger issues (for instance) when trying to > link syslinux with gnu-efi, that was built using the target toolchain. > > The first one forces ld to output i386 binary when processing the > binary files generated by nasm (for consistency with the other .o > files), and the second one explicitely uses the host toolchain to > build the utilities under utils/. > > Those two patches turned out to be necessary in order to have a > proper build. please consider them for upstream inclusion. > > There was an aditionnal third patch [1] I required, but that one isn't > authored by me. I would be grateful if you integrate it in your next > release as well. > > Best Regards, > Ben. > > [0] https://buildroot.org > [1] > https://bugs.debian.org/cgi-bin/bugreport.cgi?att=1;bug=846679;filename=syslinux_6.03%2Bdfsg-14.1.debdiff;msg=10 >I identified three categories of binaries built when building syslinux: 1. utilities required during the build (prepcore) 2. utilities meant to be run in the target operating-system (isohybrid, memdisk, the extlinux installer, ...) 3. utilities used to boot the operating system 1. and 2. can possibly run on very different systems (different libc for instance), so at least there, a distinction has to be possible. The toolchain used to build 3. has to be the same as the one you used when buidling the dependencies (gnu-efi), which is most likely the one of the target-system. While we have no issues with different libc there (no dependency on them), the version of the tools used has to be the same: you can't mix different ld version very well. My first patch is actually unrelated to that problematic and fixes an issue when building with a toolchain that defaults to 64-bits. In that case, we have to force ld to convert the nasm binary format into 32-bits elf binaries (else it generates 64-bits elf binaries). My second patch uses variables familiar to people using autoconf. I agree that a fallback should be made if those variables are not present. As it turned out, overriding the CC make variable is error-prone, so in the meantime, I have a new version of that patch that does it differently. I will send it once we agree on the need for a fix in that direction. I have a third patch that builds prepcore using the CC_FOR_BUILD compiler as well. That one was forgotten in my first serie. If you look inside the file ``mk/build.mk``, there is already a comment that mention that the distinction between the build toolchain and the host toolchain should be made. Can we at least agree that some step forward have to be made there ? Regards, Ben. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 473 bytes Desc: OpenPGP digital signature URL: <http://www.zytor.com/pipermail/syslinux/attachments/20170410/9d31427a/attachment.sig>
> Replying to my own mail as I subscribe to the ML after the answer came, > and thus never received it in my mailbox. >(snip)> > Regards, > Ben. >Since you mentioned a prior reply, here is the link, just for reference: http://www.syslinux.org/archives/2017-April/025749.html Regarding the gnu-efi submodule, I recall past questions/suggestions, requesting to use the host version of gnu-efi when building Syslinux. The answer from Syslinux developers back then was that it could generate unpredictable behavior, difficult to replicate / track down. On the other hand, there have been requests to update the gnu-efi submodule in Syslinux, so as to use an updated version / commit of it. I don't want to waste time reciting the reasons / circumstances, but this has not happened either (yet?); let's see some relevant reply to this email thread from a Syslinux developer. FWIW, I referenced the same FTBFS issue in Debian a couple of months ago: "ldlinux.elf: Not enough room for program headers" http://www.syslinux.org/archives/2017-February/025531.html with no reply, to this day. Another FWIW, there is: http://www.syslinux.org/wiki/index.php/Building @Beno?t Allard, IMHO, publicly posting up-to-date patches, explicitly and clearly expressing the commit / pre-release version being used as base, can be helpful at some point, even if Syslinux's developers don't reply immediately / on time. This might be even more important when we are talking about building-related patches. TIA, Ady.