llvm dev - Aug 2016 - Should lld support binary output ("--oformat binary")?

One remaining blocking issue for linking FreeBSD with lld is the
inability to build the bootloader components. We have a patch[1] in
review to address one of the problems by switching to using a linker
script instead of GNU ld's -N option. Another issue is that some
bootloader components are created by the linker directly as binary
objects, using --oformat binary. (Other bootloader components are
built by creating a temporary ELF object converting that to a raw
binary using objcopy.)

Is binary output enough of an unusual use case that we're unlikely to
support it in lld?

[1] https://reviews.freebsd.org/D7409

On Thu, Aug 18, 2016 at 2:16 PM, Ed Maste via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> One remaining blocking issue for linking FreeBSD with lld is the
> inability to build the bootloader components. We have a patch[1] in
> review to address one of the problems by switching to using a linker
> script instead of GNU ld's -N option. Another issue is that some
> bootloader components are created by the linker directly as binary
> objects, using --oformat binary. (Other bootloader components are
> built by creating a temporary ELF object converting that to a raw
> binary using objcopy.)
>
> Is binary output enough of an unusual use case that we're unlikely to
> support it in lld?
>
On a related note. FreeBSD (and also a product on which I work on) uses
-b for kernel modules. I really would like to avoid changing all the
invocations to objcopy. Hw vendors (e.g. network drivers which ship
with firmware) rely on this feature. With that in mind, I'm not
exactly looking forward to make them unhappy and force to change their
internal build, in particular considering how hard is to get driver
support these days.

-- 
Davide

"There are no solved problems; there are only problems that are more
or less solved" -- Henri Poincare

Maybe we should implement it. I had a concern about adding complexity for
the less frequently used feature, but I believe it can be implemented
easily. Specifically, I think we need to

 - implement another version of `Writer<ELFT>::writeSections` for
--oformat=binary and
 - stop calling `Writer::writeHeaders` and `writeBuildId` if
--oformat=binary is specified.

On Thu, Aug 18, 2016 at 8:32 AM, Davide Italiano via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On Thu, Aug 18, 2016 at 2:16 PM, Ed Maste via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
> > One remaining blocking issue for linking FreeBSD with lld is the
> > inability to build the bootloader components. We have a patch[1] in
> > review to address one of the problems by switching to using a linker
> > script instead of GNU ld's -N option. Another issue is that some
> > bootloader components are created by the linker directly as binary
> > objects, using --oformat binary. (Other bootloader components are
> > built by creating a temporary ELF object converting that to a raw
> > binary using objcopy.)
> >
> > Is binary output enough of an unusual use case that we're unlikely
to
> > support it in lld?
> >
>
> On a related note. FreeBSD (and also a product on which I work on) uses
> -b for kernel modules. I really would like to avoid changing all the
> invocations to objcopy. Hw vendors (e.g. network drivers which ship
> with firmware) rely on this feature. With that in mind, I'm not
> exactly looking forward to make them unhappy and force to change their
> internal build, in particular considering how hard is to get driver
> support these days.
>
> --
> Davide
>
> "There are no solved problems; there are only problems that are more
> or less solved" -- Henri Poincare
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160818/efcc0f84/attachment.html>

On Thu, Aug 18, 2016 at 8:32 AM, Davide Italiano via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On Thu, Aug 18, 2016 at 2:16 PM, Ed Maste via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
> > One remaining blocking issue for linking FreeBSD with lld is the
> > inability to build the bootloader components. We have a patch[1] in
> > review to address one of the problems by switching to using a linker
> > script instead of GNU ld's -N option. Another issue is that some
> > bootloader components are created by the linker directly as binary
> > objects, using --oformat binary. (Other bootloader components are
> > built by creating a temporary ELF object converting that to a raw
> > binary using objcopy.)
> >
> > Is binary output enough of an unusual use case that we're unlikely
to
> > support it in lld?
> >
>
> On a related note. FreeBSD (and also a product on which I work on) uses
> -b for kernel modules. I really would like to avoid changing all the
> invocations to objcopy. Hw vendors (e.g. network drivers which ship
> with firmware) rely on this feature. With that in mind, I'm not
> exactly looking forward to make them unhappy and force to change their
> internal build, in particular considering how hard is to get driver
> support these days.
>
This is the next issue that Michael and I ran into after figuring out what
was needed to get the kernel to boot (the remaining patches for that are in
review).

It turns out that implementing that is very simple (essentially identical
to the shader binary stuff that we already implemented for PS4, which
internally uses the same mechanism). Yesterday Michael ported that code
onto ToT and hopefully there will be a patch up soon.

-- Sean Silva

>
> --
> Davide
>
> "There are no solved problems; there are only problems that are more
> or less solved" -- Henri Poincare
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160818/5a7bbe83/attachment.html>