llvm dev - Jun 2019 - Need help on identifying a patch which fixed lld on linux platform

Hi All,

This test is on a ubuntu 12 box. Can anyone please point me what
revision/commit-id of lld fixed this issue which was atleast in llvm 5.0.
.
├── build.sh
├── main.c
├── shared
│   └── sh.c
└── static
    └── st.c

[[ build.sh ]]

#!/bin/sh

CC="${TOOLCHAIN}/bin/clang"
AR="${TOOLCHAIN}/bin/llvm-ar"
CFLAGS="-g -O"
LDFLAGS="-fuse-ld=lld"

rm shared/sh.o static/st.o main.o
rm shared/libsh.so static/libst.a main

${CC} ${CFLAGS} -c shared/sh.c -o shared/sh.o
${CC} ${CFLAGS} ${LDFLAGS} -shared -o shared/libsh.so shared/sh.o

${CC} ${CFLAGS} -c static/st.c -o static/st.o
${AR} cq static/libst.a static/st.o

${CC} ${CFLAGS} -c main.c -o main.o

${CC} ${CFLAGS} ${LDFLAGS} -o main -Lshared -lsh -Lstatic -lst  main.o
-Wl,-rpath=shared

[[ main.c ]]

void gn();

int main()
{
    gn();
    return 0;
}

[[ shared/sh.c ]]

#include <stdio.h>

void gn(void);
void fn(void);

void gn()
{
    printf("Calling gn...\n");
    fn();
}

[[ static/st.c ]]

#include <stdio.h>

void fn(void);

void fn()
{
    printf("Calling fn...\n");
}


Code flow:
main -> gn (shared library) -> fn (part of the static lib)

Result:
With llvm 5.0

./main
Calling gn...
./main: symbol lookup error: shared/libsh.so: undefined symbol: fn   <=symbol
fn was not found in binary "main"

readelf -s main | grep fn
<Nothing>

With llvm 7.0

./main
Calling gn...
Calling fn...

readelf -s main | grep fn
     9: 0000000000001100    12 FUNC    GLOBAL DEFAULT   13 fn
    36: 0000000000001100    12 FUNC    GLOBAL DEFAULT   13 fn

Regards,
Karan
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190613/5cc32aba/attachment.html>

Looks like Ubuntu 12 was released in 2012, and the most recent version of
LLVM is LLVM 8.0.0.

LLVM 5.0 is pretty old, and in particular, lld in LLVM 5.0 is extremely
outdated. IIUC, the first release of LLVM that includes somewhat usable
version of lld is LLVM 4.0, and I wouldn't be surprised that LLVM 5.0 has a
lot of bugs.

Can't you simply use a newer version of lld?

On Thu, Jun 13, 2019 at 4:01 PM karnajit wangkhem via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi All,
>
> This test is on a ubuntu 12 box. Can anyone please point me what
> revision/commit-id of lld fixed this issue which was atleast in llvm 5.0.
> .
> ├── build.sh
> ├── main.c
> ├── shared
> │   └── sh.c
> └── static
>     └── st.c
>
> [[ build.sh ]]
>
> #!/bin/sh
>
> CC="${TOOLCHAIN}/bin/clang"
> AR="${TOOLCHAIN}/bin/llvm-ar"
> CFLAGS="-g -O"
> LDFLAGS="-fuse-ld=lld"
>
> rm shared/sh.o static/st.o main.o
> rm shared/libsh.so static/libst.a main
>
> ${CC} ${CFLAGS} -c shared/sh.c -o shared/sh.o
> ${CC} ${CFLAGS} ${LDFLAGS} -shared -o shared/libsh.so shared/sh.o
>
> ${CC} ${CFLAGS} -c static/st.c -o static/st.o
> ${AR} cq static/libst.a static/st.o
>
> ${CC} ${CFLAGS} -c main.c -o main.o
>
> ${CC} ${CFLAGS} ${LDFLAGS} -o main -Lshared -lsh -Lstatic -lst  main.o
> -Wl,-rpath=shared
>
> [[ main.c ]]
>
> void gn();
>
> int main()
> {
>     gn();
>     return 0;
> }
>
> [[ shared/sh.c ]]
>
> #include <stdio.h>
>
> void gn(void);
> void fn(void);
>
> void gn()
> {
>     printf("Calling gn...\n");
>     fn();
> }
>
> [[ static/st.c ]]
>
> #include <stdio.h>
>
> void fn(void);
>
> void fn()
> {
>     printf("Calling fn...\n");
> }
>
>
> Code flow:
> main -> gn (shared library) -> fn (part of the static lib)
>
> Result:
> With llvm 5.0
>
> ./main
> Calling gn...
> ./main: symbol lookup error: shared/libsh.so: undefined symbol: fn  
<=> symbol fn was not found in binary "main"
>
> readelf -s main | grep fn
> <Nothing>
>
> With llvm 7.0
>
> ./main
> Calling gn...
> Calling fn...
>
> readelf -s main | grep fn
>      9: 0000000000001100    12 FUNC    GLOBAL DEFAULT   13 fn
>     36: 0000000000001100    12 FUNC    GLOBAL DEFAULT   13 fn
>
> Regards,
> Karan
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190613/07d2d14d/attachment.html>

Thanks for the info Rui.

Transitioning from llvm 5.0 to llvm 7.0 will take some time due to the
nature/process of the production environment. Is it ok to use lld 7.0 with
llvm 5.0?

On Thu, Jun 13, 2019 at 1:47 PM Rui Ueyama <ruiu at google.com> wrote:
> Looks like Ubuntu 12 was released in 2012, and the most recent version of
> LLVM is LLVM 8.0.0.
>
> LLVM 5.0 is pretty old, and in particular, lld in LLVM 5.0 is extremely
> outdated. IIUC, the first release of LLVM that includes somewhat usable
> version of lld is LLVM 4.0, and I wouldn't be surprised that LLVM 5.0
has a
> lot of bugs.
>
> Can't you simply use a newer version of lld?
>
> On Thu, Jun 13, 2019 at 4:01 PM karnajit wangkhem via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi All,
>>
>> This test is on a ubuntu 12 box. Can anyone please point me what
>> revision/commit-id of lld fixed this issue which was atleast in llvm
5.0.
>> .
>> ├── build.sh
>> ├── main.c
>> ├── shared
>> │   └── sh.c
>> └── static
>>     └── st.c
>>
>> [[ build.sh ]]
>>
>> #!/bin/sh
>>
>> CC="${TOOLCHAIN}/bin/clang"
>> AR="${TOOLCHAIN}/bin/llvm-ar"
>> CFLAGS="-g -O"
>> LDFLAGS="-fuse-ld=lld"
>>
>> rm shared/sh.o static/st.o main.o
>> rm shared/libsh.so static/libst.a main
>>
>> ${CC} ${CFLAGS} -c shared/sh.c -o shared/sh.o
>> ${CC} ${CFLAGS} ${LDFLAGS} -shared -o shared/libsh.so shared/sh.o
>>
>> ${CC} ${CFLAGS} -c static/st.c -o static/st.o
>> ${AR} cq static/libst.a static/st.o
>>
>> ${CC} ${CFLAGS} -c main.c -o main.o
>>
>> ${CC} ${CFLAGS} ${LDFLAGS} -o main -Lshared -lsh -Lstatic -lst  main.o
>> -Wl,-rpath=shared
>>
>> [[ main.c ]]
>>
>> void gn();
>>
>> int main()
>> {
>>     gn();
>>     return 0;
>> }
>>
>> [[ shared/sh.c ]]
>>
>> #include <stdio.h>
>>
>> void gn(void);
>> void fn(void);
>>
>> void gn()
>> {
>>     printf("Calling gn...\n");
>>     fn();
>> }
>>
>> [[ static/st.c ]]
>>
>> #include <stdio.h>
>>
>> void fn(void);
>>
>> void fn()
>> {
>>     printf("Calling fn...\n");
>> }
>>
>>
>> Code flow:
>> main -> gn (shared library) -> fn (part of the static lib)
>>
>> Result:
>> With llvm 5.0
>>
>> ./main
>> Calling gn...
>> ./main: symbol lookup error: shared/libsh.so: undefined symbol: fn  
<=>> symbol fn was not found in binary "main"
>>
>> readelf -s main | grep fn
>> <Nothing>
>>
>> With llvm 7.0
>>
>> ./main
>> Calling gn...
>> Calling fn...
>>
>> readelf -s main | grep fn
>>      9: 0000000000001100    12 FUNC    GLOBAL DEFAULT   13 fn
>>     36: 0000000000001100    12 FUNC    GLOBAL DEFAULT   13 fn
>>
>> Regards,
>> Karan
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190613/043f89fa/attachment.html>

On 13 Jun 2019, at 08:59, karnajit wangkhem via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> This test is on a ubuntu 12 box. Can anyone please point me what
revision/commit-id of lld fixed this issue which was atleast in llvm 5.0.
...
> Result:
> With llvm 5.0
> 
> ./main
> Calling gn...
> ./main: symbol lookup error: shared/libsh.so: undefined symbol: fn   <==
symbol fn was not found in binary "main"
This was fixed in https://reviews.llvm.org/rLLD325849, after 6.0.0 was released:

------------------------------------------------------------------------
r325849 | ruiu | 2018-02-23 01:16:57 +0000 (Fri, 23 Feb 2018) | 16 lines

Make undefined symbol in DSO to pull out object files from archive files.

We have an internal program that does't link without this patch. I don't
know of any open-source program that needs this, but there might be.
Since this patch improves compatibility with GNU linkers with a few lines
of code, I think it's worth to be committed.

The problem is about undefined symbols in DSOs. Some programs depend on
the GNU linkers' behavior that they pull out object files from archive
files to resolve undefined symbols in DSOs. We already allow that kind of
"reverse" dependency (from DSOs to the main executable) for regular
symbols, in particular, for "__progname" symbol (which is usually in
crt0.o), but that doesn't work if the symbol is in an archive file.
This patch is to make it work.

Differential Revision: https://reviews.llvm.org/D43658
------------------------------------------------------------------------

-Dimitry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 223 bytes
Desc: Message signed with OpenPGP
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190613/a2a0e322/attachment.sig>

Thanks a lot for the pointer to this patch.

Regards,
Karan



On Thu, Jun 13, 2019, 4:17 PM Dimitry Andric <dimitry at andric.com>
wrote:
> On 13 Jun 2019, at 08:59, karnajit wangkhem via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >
> > This test is on a ubuntu 12 box. Can anyone please point me what
> revision/commit-id of lld fixed this issue which was atleast in llvm 5.0.
> ...
> > Result:
> > With llvm 5.0
> >
> > ./main
> > Calling gn...
> > ./main: symbol lookup error: shared/libsh.so: undefined symbol: fn  
<=> symbol fn was not found in binary "main"
>
> This was fixed in https://reviews.llvm.org/rLLD325849, after 6.0.0 was
> released:
>
> ------------------------------------------------------------------------
> r325849 | ruiu | 2018-02-23 01:16:57 +0000 (Fri, 23 Feb 2018) | 16 lines
>
> Make undefined symbol in DSO to pull out object files from archive files.
>
> We have an internal program that does't link without this patch. I
don't
> know of any open-source program that needs this, but there might be.
> Since this patch improves compatibility with GNU linkers with a few lines
> of code, I think it's worth to be committed.
>
> The problem is about undefined symbols in DSOs. Some programs depend on
> the GNU linkers' behavior that they pull out object files from archive
> files to resolve undefined symbols in DSOs. We already allow that kind of
> "reverse" dependency (from DSOs to the main executable) for
regular
> symbols, in particular, for "__progname" symbol (which is usually
in
> crt0.o), but that doesn't work if the symbol is in an archive file.
> This patch is to make it work.
>
> Differential Revision: https://reviews.llvm.org/D43658
> ------------------------------------------------------------------------
>
> -Dimitry
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190613/b4eb8887/attachment.html>