llvm dev - Sep 2016 - Commit zlib source code into to llvm repo

Yes, zlib build for LTO will work, but user will have to build this lib
from zlib source code anyway.

On Fri, Sep 23, 2016 at 4:01 PM Mehdi Amini <mehdi.amini at apple.com>
wrote:
>
> On Sep 23, 2016, at 1:44 PM, Vitaly Buka via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> Hi all,
>
> Is it possible to add zlib source code into llvm repo? Any advice on how
> to do so?
>
> I'd like to setup build of symbolizer as a static library with no
external
> dependencies except libc.
> Similar to
>
https://github.com/google/sanitizers/blob/master/address-sanitizer/internal_symbolizer/howto
> .
> Then the library can be linked into sanitized program and avoid dependency
> on standalone llvm-symbolizer tool.
>
> Out solution uses LTO and internalize everything but symbolizer API, so we
> have no symbol conflicts with instrumented binary.
> For LTO we need zlib in source code.
>
>
> This is not clear to me.
>
> It seems that what you want ultimately is a build where zlib would be LTO
> together with your program?
> Assuming this is correct, you don’t need the source code for zlib, but you
> need a static archive of zlib build for LTO.
> (What may be missing, is the ability to the LLVM build system to select a
> static zlib archive to link to?)
>
> —
> Mehdi
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160923/012a0bc3/attachment.html>

Hi Vitaly,

Can you go over the exact use case with a bit more details? (i.e. without
assuming we’re all familiar with the product, who are the user, and how are they
using it).
That’d help to understand the need dump an external library source code directly
inside LLVM.

Also have you consider changing the LLVM build system to build zlib optionally
as an external project? (Like we can include clang -DLLVM_EXTERNAL_CLANG_DIR=….)

— 
Mehdi


> On Sep 23, 2016, at 4:35 PM, Vitaly Buka <vitalybuka at google.com>
wrote:
> 
> Yes, zlib build for LTO will work, but user will have to build this lib
from zlib source code anyway.
> 
> On Fri, Sep 23, 2016 at 4:01 PM Mehdi Amini <mehdi.amini at apple.com
<mailto:mehdi.amini at apple.com>> wrote:
> 
>> On Sep 23, 2016, at 1:44 PM, Vitaly Buka via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>> 
>> Hi all,
>> 
>> Is it possible to add zlib source code into llvm repo? Any advice on
how to do so?
>> 
>> I'd like to setup build of symbolizer as a static library with no
external dependencies except libc.
>> Similar to
https://github.com/google/sanitizers/blob/master/address-sanitizer/internal_symbolizer/howto
<https://github.com/google/sanitizers/blob/master/address-sanitizer/internal_symbolizer/howto>.
>> Then the library can be linked into sanitized program and avoid
dependency on standalone llvm-symbolizer tool.
>> 
>> Out solution uses LTO and internalize everything but symbolizer API, so
we have no symbol conflicts with instrumented binary.
>> For LTO we need zlib in source code.
> 
> This is not clear to me.
> 
> It seems that what you want ultimately is a build where zlib would be LTO
together with your program?
> Assuming this is correct, you don’t need the source code for zlib, but you
need a static archive of zlib build for LTO.
> (What may be missing, is the ability to the LLVM build system to select a
static zlib archive to link to?)
> 
> — 
> Mehdi
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160923/ab63bab3/attachment.html>

On Fri, Sep 23, 2016 at 4:58 PM Mehdi Amini <mehdi.amini at apple.com>
wrote:
> Hi Vitaly,
>
> Can you go over the exact use case with a bit more details? (i.e. without
> assuming we’re all familiar with the product, who are the user, and how are
> they using it).
> That’d help to understand the need dump an external library source code
> directly inside LLVM.
>
Sanitizer runtime supports weak hooks for symbolizer interface.
http://llvm-cs.pcc.me.uk/projects/compiler-rt/lib/sanitizer_common/sanitizer_symbolizer_posix_libcdep.cc#328
If these symbols are defined, runtime will use them instead of invoking
llvm-symbolizer tool.
So it would be nice to have this library in clang distribution and make
clang driver link this library with sanitizer runtimes.

Library includes llvm::symbolize::LLVMSymbolizer with dependencies, mostly
from LLVM plus zlib.
Library must not be instrumented with sanitizes and do not depend on any
instrumented code. So using LTO and -internalize-public-api-list flag we
make sure that library exports only weak hooks like: __sanitizer_symbolize_*
<http://llvm-cs.pcc.me.uk/projects/compiler-rt/lib/sanitizer_common/sanitizer_symbolizer_posix_libcdep.cc/r__sanitizer_symbolize_demangle>
symbols,
and has undefined symbols only from libc.

>
> Also have you consider changing the LLVM build system to build zlib
> optionally as an external project? (Like we can include clang
> -DLLVM_EXTERNAL_CLANG_DIR=….)
>
Maybe -DLLVM_EXTERNAL_ZLIB is OK temporarily, but long temp it does not
sound like convenient solution.

>
> —
> Mehdi
>
>
>
> On Sep 23, 2016, at 4:35 PM, Vitaly Buka <vitalybuka at google.com>
wrote:
>
> Yes, zlib build for LTO will work, but user will have to build this lib
> from zlib source code anyway.
>
> On Fri, Sep 23, 2016 at 4:01 PM Mehdi Amini <mehdi.amini at
apple.com> wrote:
>
>
> On Sep 23, 2016, at 1:44 PM, Vitaly Buka via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> Hi all,
>
> Is it possible to add zlib source code into llvm repo? Any advice on how
> to do so?
>
> I'd like to setup build of symbolizer as a static library with no
external
> dependencies except libc.
> Similar to
>
https://github.com/google/sanitizers/blob/master/address-sanitizer/internal_symbolizer/howto
> .
> Then the library can be linked into sanitized program and avoid dependency
> on standalone llvm-symbolizer tool.
>
> Out solution uses LTO and internalize everything but symbolizer API, so we
> have no symbol conflicts with instrumented binary.
> For LTO we need zlib in source code.
>
>
> This is not clear to me.
>
> It seems that what you want ultimately is a build where zlib would be LTO
> together with your program?
> Assuming this is correct, you don’t need the source code for zlib, but you
> need a static archive of zlib build for LTO.
> (What may be missing, is the ability to the LLVM build system to select a
> static zlib archive to link to?)
>
> —
> Mehdi
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160924/01268205/attachment.html>