llvm dev - Apr 2021 - Porting libxar to LLVM

Hi,

I want to float the idea of rewriting this external dependency as part
of LLVM. For context, Apple's `-bitcode_bundle` flag packages bitcode
within the `__LLVM` segment of the larger executable. The bitcode
files are packaged by serializing them into a XAR (XML Archive) file,
which then gets copied into `__LLVM`. llvm-objdump and LLD use libxar
to read and write these XARs.

I think there are several good reasons for writing our own libxar:
1. libxar's interface doesn't allow for *not* writing to disk (i.e. we
cannot write to an mmap'ed region), so it incurs unnecessary I/O
overhead
2. It doesn't support Windows
3. It exposes a rather primitive C interface (using raw ints to
indicate error status) that is out of place in LLVM's C++

The goal would be to have both LLD and llvm-objdump use this
home-grown implementation of libxar, instead of the system one.

One possibility is to contribute to the existing libxar, but I don't
know if it is really maintained; Apple releases open source tarball
drops every so often, and there's https://mackyle.github.io/xar/ which
appears to be a fork of the now-defunct http://xar.googlecode.com/,
but even that hasn't seen a commit since 2014.

I may have an undergrad who's eager to take this work on, but before
they embark down that path I wanted to run it by y'all. Let me know if
you have thoughts.

Cheers,
Jez

Might want to CC any of the original authors from Apple - I expect
they'd have some thoughts on this, maybe know something about the
maintained-ness status of libxar, etc.

If we do end up writing something of our own, I wonder if it has to be
xar or whether there's something else we should entertain? (maybe the
bitcode format could be reused/broadened/or the like)

On Thu, Apr 29, 2021 at 7:51 PM Jez via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> Hi,
>
> I want to float the idea of rewriting this external dependency as part
> of LLVM. For context, Apple's `-bitcode_bundle` flag packages bitcode
> within the `__LLVM` segment of the larger executable. The bitcode
> files are packaged by serializing them into a XAR (XML Archive) file,
> which then gets copied into `__LLVM`. llvm-objdump and LLD use libxar
> to read and write these XARs.
>
> I think there are several good reasons for writing our own libxar:
> 1. libxar's interface doesn't allow for *not* writing to disk (i.e.
we
> cannot write to an mmap'ed region), so it incurs unnecessary I/O
> overhead
> 2. It doesn't support Windows
> 3. It exposes a rather primitive C interface (using raw ints to
> indicate error status) that is out of place in LLVM's C++
>
> The goal would be to have both LLD and llvm-objdump use this
> home-grown implementation of libxar, instead of the system one.
>
> One possibility is to contribute to the existing libxar, but I don't
> know if it is really maintained; Apple releases open source tarball
> drops every so often, and there's https://mackyle.github.io/xar/ which
> appears to be a fork of the now-defunct http://xar.googlecode.com/,
> but even that hasn't seen a commit since 2014.
>
> I may have an undergrad who's eager to take this work on, but before
> they embark down that path I wanted to run it by y'all. Let me know if
> you have thoughts.
>
> Cheers,
> Jez
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev