llvm dev - Nov 2018 - A stage2 build causes changes to libllvm impacting program using it (exemple: rustc)

Hello,

Lately, I have been working on moving Debian & Ubuntu packages to a
stage2 build.

This means that, instead of shipping llvm-toolchain packages built with
gcc, we are rebuilding
everything a second time using the newly built clang.

Now, when pushed to Debian, it caused some unexpected issues in
particular with rust reported here:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=913271#35

rustc in Debian was building initially against libllvm built with gcc.
It was working fine.
The failure is now happening when the systems saw their libllvm upgraded
to the stage2 bootstrap one.

The beginning of the backtrace is the following (generating debug info):

Thread 1 "rustc" received signal SIGSEGV, Segmentation fault.
0x00007ffff1e273bc in llvm::StringMapImpl::LookupBucketFor(llvm::StringRef) ()
from /usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
(gdb) bt
#0  0x00007ffff1e273bc in llvm::StringMapImpl::LookupBucketFor(llvm::StringRef)
() from /usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
#1  0x00007ffff1f654c9 in ?? () from /usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
#2  0x00007ffff1f65491 in llvm::MDString::get(llvm::LLVMContext&,
llvm::StringRef) () from /usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
#3  0x00007ffff1ee69cd in ?? () from /usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
#4  0x00007ffff1ee2d12 in llvm::DIBuilder::createFile(llvm::StringRef,
llvm::StringRef,
llvm::Optional<llvm::DIFile::ChecksumInfo<llvm::StringRef> >,
llvm::Optional<llvm::StringRef>) () from
/usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
#5  0x00007ffff50a01f2 in LLVMRustDIBuilderCreateFile (Builder=0x5555555d4180,
Filename=0x7ffffffeedd8 "src/main.rs", Directory=0x555555b3dd10
"$HOME/hello_world")
    at /usr/lib/llvm-7/include/llvm/ADT/StringRef.h:85
#6  0x00007ffff501d39e in
rustc_codegen_llvm::debuginfo::metadata::compile_unit_metadata (tcx=...,
codegen_unit_name=..., debug_context=0x7ffffffeeeb0) at
librustc_codegen_llvm/debuginfo/metadata.rs:859

Disabling the debug generation for rustc fixes the issue (thanks to Ximin Luo
for the investigation).

Why idea how to workaround this issue ?

Unfortunately, because rustc needs rustc to build itself, I cannot check if a
simple recompilation fixes it or not.

Cheers,
S

I don't know about the real issue here, but you should be able to test if a
rebuild of rustc fixes it, by changing the soversion of the newly built
libLLVM-7 (and others, if relevant). Then you can leave the old libs on
your system for the existing rustc binary, and have the new build use the
new one.

That doesn't seem likely to help -- unless you're also changing some
headers rustc is compiling against? But worth testing I suppose.


On Sun, Nov 11, 2018, 3:54 AM Sylvestre Ledru via llvm-dev <
llvm-dev at lists.llvm.org wrote:
> Hello,
>
> Lately, I have been working on moving Debian & Ubuntu packages to a
> stage2 build.
>
> This means that, instead of shipping llvm-toolchain packages built with
> gcc, we are rebuilding
> everything a second time using the newly built clang.
>
> Now, when pushed to Debian, it caused some unexpected issues in
> particular with rust reported here:
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=913271#35
>
> rustc in Debian was building initially against libllvm built with gcc.
> It was working fine.
> The failure is now happening when the systems saw their libllvm upgraded
> to the stage2 bootstrap one.
>
> The beginning of the backtrace is the following (generating debug info):
>
> Thread 1 "rustc" received signal SIGSEGV, Segmentation fault.
> 0x00007ffff1e273bc in
> llvm::StringMapImpl::LookupBucketFor(llvm::StringRef) () from
> /usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
> (gdb) bt
> #0  0x00007ffff1e273bc in
> llvm::StringMapImpl::LookupBucketFor(llvm::StringRef) () from
> /usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
> #1  0x00007ffff1f654c9 in ?? () from
> /usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
> #2  0x00007ffff1f65491 in llvm::MDString::get(llvm::LLVMContext&,
> llvm::StringRef) () from /usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
> #3  0x00007ffff1ee69cd in ?? () from
> /usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
> #4  0x00007ffff1ee2d12 in llvm::DIBuilder::createFile(llvm::StringRef,
> llvm::StringRef,
llvm::Optional<llvm::DIFile::ChecksumInfo<llvm::StringRef>
> >, llvm::Optional<llvm::StringRef>) () from
> /usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
> #5  0x00007ffff50a01f2 in LLVMRustDIBuilderCreateFile
> (Builder=0x5555555d4180, Filename=0x7ffffffeedd8 "src/main.rs",
> Directory=0x555555b3dd10 "$HOME/hello_world")
>     at /usr/lib/llvm-7/include/llvm/ADT/StringRef.h:85
> #6  0x00007ffff501d39e in
> rustc_codegen_llvm::debuginfo::metadata::compile_unit_metadata (tcx=...,
> codegen_unit_name=..., debug_context=0x7ffffffeeeb0) at
> librustc_codegen_llvm/debuginfo/metadata.rs:859
>
> Disabling the debug generation for rustc fixes the issue (thanks to Ximin
> Luo for the investigation).
>
> Why idea how to workaround this issue ?
>
> Unfortunately, because rustc needs rustc to build itself, I cannot check
> if a simple recompilation fixes it or not.
>
> Cheers,
> S
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20181111/a300bd64/attachment.html>

Hi Sylvestre,

isn't this another instance of the clang-7/gcc ABI issue around Optional?

llvm::DIBuilder::createFile accepts Optional arguments, and more than once
people complained to me or on IRC about that particular call segfaulting.

Cheers,
Philip

On Sun, Nov 11, 2018 at 9:54 AM Sylvestre Ledru via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hello,
>
> Lately, I have been working on moving Debian & Ubuntu packages to a
> stage2 build.
>
> This means that, instead of shipping llvm-toolchain packages built with
> gcc, we are rebuilding
> everything a second time using the newly built clang.
>
> Now, when pushed to Debian, it caused some unexpected issues in
> particular with rust reported here:
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=913271#35
>
> rustc in Debian was building initially against libllvm built with gcc.
> It was working fine.
> The failure is now happening when the systems saw their libllvm upgraded
> to the stage2 bootstrap one.
>
> The beginning of the backtrace is the following (generating debug info):
>
> Thread 1 "rustc" received signal SIGSEGV, Segmentation fault.
> 0x00007ffff1e273bc in
> llvm::StringMapImpl::LookupBucketFor(llvm::StringRef) () from
> /usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
> (gdb) bt
> #0  0x00007ffff1e273bc in
> llvm::StringMapImpl::LookupBucketFor(llvm::StringRef) () from
> /usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
> #1  0x00007ffff1f654c9 in ?? () from
> /usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
> #2  0x00007ffff1f65491 in llvm::MDString::get(llvm::LLVMContext&,
> llvm::StringRef) () from /usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
> #3  0x00007ffff1ee69cd in ?? () from
> /usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
> #4  0x00007ffff1ee2d12 in llvm::DIBuilder::createFile(llvm::StringRef,
> llvm::StringRef,
llvm::Optional<llvm::DIFile::ChecksumInfo<llvm::StringRef>
> >, llvm::Optional<llvm::StringRef>) () from
> /usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
> #5  0x00007ffff50a01f2 in LLVMRustDIBuilderCreateFile
> (Builder=0x5555555d4180, Filename=0x7ffffffeedd8 "src/main.rs",
> Directory=0x555555b3dd10 "$HOME/hello_world")
>     at /usr/lib/llvm-7/include/llvm/ADT/StringRef.h:85
> #6  0x00007ffff501d39e in
> rustc_codegen_llvm::debuginfo::metadata::compile_unit_metadata (tcx=...,
> codegen_unit_name=..., debug_context=0x7ffffffeeeb0) at
> librustc_codegen_llvm/debuginfo/metadata.rs:859
>
> Disabling the debug generation for rustc fixes the issue (thanks to Ximin
> Luo for the investigation).
>
> Why idea how to workaround this issue ?
>
> Unfortunately, because rustc needs rustc to build itself, I cannot check
> if a simple recompilation fixes it or not.
>
> Cheers,
> S
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20181111/d922bd85/attachment.html>

I don't know yet. Would you have a pointer which explains how to fix the
issue?

Thanks
Sylvestre

Le 11/11/2018 à 20:34, Philip Pfaffe a écrit :
> Hi Sylvestre,
> 
> isn't this another instance of the clang-7/gcc ABI issue around
Optional?
> 
> llvm::DIBuilder::createFile accepts Optional arguments, and more than once
people complained to me or on IRC about that particular call segfaulting.
> 
> Cheers,
> Philip
> 
> On Sun, Nov 11, 2018 at 9:54 AM Sylvestre Ledru via llvm-dev <llvm-dev
at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> 
>     Hello,
> 
>     Lately, I have been working on moving Debian & Ubuntu packages to a
>     stage2 build.
> 
>     This means that, instead of shipping llvm-toolchain packages built with
>     gcc, we are rebuilding
>     everything a second time using the newly built clang.
> 
>     Now, when pushed to Debian, it caused some unexpected issues in
>     particular with rust reported here:
>     https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=913271#35
> 
>     rustc in Debian was building initially against libllvm built with gcc.
>     It was working fine.
>     The failure is now happening when the systems saw their libllvm
upgraded
>     to the stage2 bootstrap one.
> 
>     The beginning of the backtrace is the following (generating debug
info):
> 
>     Thread 1 "rustc" received signal SIGSEGV, Segmentation fault.
>     0x00007ffff1e273bc in
llvm::StringMapImpl::LookupBucketFor(llvm::StringRef) () from
/usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
>     (gdb) bt
>     #0  0x00007ffff1e273bc in
llvm::StringMapImpl::LookupBucketFor(llvm::StringRef) () from
/usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
>     #1  0x00007ffff1f654c9 in ?? () from
/usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
>     #2  0x00007ffff1f65491 in llvm::MDString::get(llvm::LLVMContext&,
llvm::StringRef) () from /usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
>     #3  0x00007ffff1ee69cd in ?? () from
/usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
>     #4  0x00007ffff1ee2d12 in llvm::DIBuilder::createFile(llvm::StringRef,
llvm::StringRef,
llvm::Optional<llvm::DIFile::ChecksumInfo<llvm::StringRef> >,
llvm::Optional<llvm::StringRef>) () from
/usr/lib/x86_64-linux-gnu/libLLVM-7.so.1
>     #5  0x00007ffff50a01f2 in LLVMRustDIBuilderCreateFile
(Builder=0x5555555d4180, Filename=0x7ffffffeedd8 "src/main.rs
<http://main.rs>", Directory=0x555555b3dd10
"$HOME/hello_world")
>         at /usr/lib/llvm-7/include/llvm/ADT/StringRef.h:85
>     #6  0x00007ffff501d39e in
rustc_codegen_llvm::debuginfo::metadata::compile_unit_metadata (tcx=...,
codegen_unit_name=..., debug_context=0x7ffffffeeeb0) at
librustc_codegen_llvm/debuginfo/metadata.rs:859 <http://metadata.rs:859>
> 
>     Disabling the debug generation for rustc fixes the issue (thanks to
Ximin Luo for the investigation).
> 
>     Why idea how to workaround this issue ?
> 
>     Unfortunately, because rustc needs rustc to build itself, I cannot
check if a simple recompilation fixes it or not.
> 
>     Cheers,
>     S
> 
> 
>     _______________________________________________
>     LLVM Developers mailing list
>     llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>     http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>