llvm dev - Nov 2014 - [LLVMdev] External names for LTO in gold plugin

Hi,

In my work applying -flto to Chrome, I need to set some names to be skipped
by the InternalizePass; otherwise, the linking stage fails (i.e., when
building the chrome binary). In the past, I had a tiny patch that I hadn't
submitted to LLVM: it was something like:

Index: Internalize.cpp
==================================================================---
Internalize.cpp     (revision 221403)
+++ Internalize.cpp     (working copy)
@@ -87,6 +87,7 @@
         itr != ExportList.end(); itr++) {
     ExternalNames.insert(*itr);
   }
+  ExternalNames.insert(APIList.begin(), APIList.end());
 }


This adds the contents of the -internalize-public-api-list flag to the set
of external names for the internalize pass, even when there are already
some names being passed in the constructor.

Then I would pass, e.g., "-Wl,-internalize-public-api-list=__xstat64"
to
the linking phase, and all would be well. However, since the gold plugin
has been changed significantly, this no longer works, since the flags from
Internalize.cpp don't get up to the gold plugin, and I don't know the
new
gold plugin bits well enough to see immediately how to replace it. I could
go spelunking for the answer, but I wanted to ask here first.

Is there currently a way for me to give the gold plugin a list of names
that shouldn't be internalized?

Thanks,

Tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20141110/c6af8a43/attachment.html>

On 10 November 2014 17:33, Tom Roeder <tmroeder at google.com>
wrote:
> Hi,
>
> In my work applying -flto to Chrome, I need to set some names to be skipped
> by the InternalizePass; otherwise, the linking stage fails (i.e., when
> building the chrome binary). In the past, I had a tiny patch that I
hadn't
> submitted to LLVM: it was something like:
>
> Index: Internalize.cpp
> ==================================================================> ---
Internalize.cpp     (revision 221403)
> +++ Internalize.cpp     (working copy)
> @@ -87,6 +87,7 @@
>          itr != ExportList.end(); itr++) {
>      ExternalNames.insert(*itr);
>    }
> +  ExternalNames.insert(APIList.begin(), APIList.end());
>  }
>
>
> This adds the contents of the -internalize-public-api-list flag to the set
> of external names for the internalize pass, even when there are already
some
> names being passed in the constructor.
>
> Then I would pass, e.g.,
"-Wl,-internalize-public-api-list=__xstat64" to the
> linking phase, and all would be well. However, since the gold plugin has
> been changed significantly, this no longer works, since the flags from
> Internalize.cpp don't get up to the gold plugin, and I don't know
the new
> gold plugin bits well enough to see immediately how to replace it. I could
> go spelunking for the answer, but I wanted to ask here first.
>
> Is there currently a way for me to give the gold plugin a list of names
that
> shouldn't be internalized?
The current plugin doesn't even use internalize. It depends on the
linker's decision of what should be kept. Given that, your best option
is convincing the linker that the symbol is needed. One way to do it
is with the -u option:

--------------------------------------------------------------
$ llvm-dis  test.o -o -
; ModuleID = 'test.o'

define i32 @f() {
  ret i32 42
}

define i32 @main() {
  ret i32 42
}
$ ./build/bin/clang  test.o -flto -o test -Wl,-plugin-opt=save-temps
$ llvm-dis test.bc -o -
; ModuleID = 'test.bc'
target triple = "x86_64-unknown-linux-gnu"

define internal i32 @f() {
  ret i32 42
}

define i32 @main() {
  ret i32 42
}
$ ./build/bin/clang  test.o -flto -o test -Wl,-plugin-opt=save-temps
-Wl,-u -Wl,f
$ llvm-dis test.bc -o -
; ModuleID = 'test.bc'
target triple = "x86_64-unknown-linux-gnu"

define i32 @f() {
  ret i32 42
}

define i32 @main() {
  ret i32 42
}
------------------------------------------------------

Note how on the second run @f is not internalized.

Cheers,
Rafael

> On 2014-Nov-10, at 14:53, Rafael Espíndola <rafael.espindola at
gmail.com> wrote:
> 
> On 10 November 2014 17:33, Tom Roeder <tmroeder at google.com> wrote:
>> Hi,
>> 
>> In my work applying -flto to Chrome, I need to set some names to be
skipped
>> by the InternalizePass; otherwise, the linking stage fails (i.e., when
>> building the chrome binary). In the past, I had a tiny patch that I
hadn't
>> submitted to LLVM: it was something like:
>> 
>> Index: Internalize.cpp
>>
==================================================================>> ---
Internalize.cpp     (revision 221403)
>> +++ Internalize.cpp     (working copy)
>> @@ -87,6 +87,7 @@
>>         itr != ExportList.end(); itr++) {
>>     ExternalNames.insert(*itr);
>>   }
>> +  ExternalNames.insert(APIList.begin(), APIList.end());
>> }
>> 
>> 
>> This adds the contents of the -internalize-public-api-list flag to the
set
>> of external names for the internalize pass, even when there are already
some
>> names being passed in the constructor.
>> 
>> Then I would pass, e.g.,
"-Wl,-internalize-public-api-list=__xstat64" to the
>> linking phase, and all would be well. However, since the gold plugin
has
>> been changed significantly, this no longer works, since the flags from
>> Internalize.cpp don't get up to the gold plugin, and I don't
know the new
>> gold plugin bits well enough to see immediately how to replace it. I
could
>> go spelunking for the answer, but I wanted to ask here first.
>> 
>> Is there currently a way for me to give the gold plugin a list of names
that
>> shouldn't be internalized?
> 
> The current plugin doesn't even use internalize. It depends on the
> linker's decision of what should be kept. Given that, your best option
> is convincing the linker that the symbol is needed. One way to do it
> is with the -u option:
> 
> --------------------------------------------------------------
> $ llvm-dis  test.o -o -
> ; ModuleID = 'test.o'
> 
> define i32 @f() {
>  ret i32 42
> }
> 
> define i32 @main() {
>  ret i32 42
> }
> $ ./build/bin/clang  test.o -flto -o test -Wl,-plugin-opt=save-temps
> $ llvm-dis test.bc -o -
> ; ModuleID = 'test.bc'
> target triple = "x86_64-unknown-linux-gnu"
> 
> define internal i32 @f() {
>  ret i32 42
> }
> 
> define i32 @main() {
>  ret i32 42
> }
> $ ./build/bin/clang  test.o -flto -o test -Wl,-plugin-opt=save-temps
> -Wl,-u -Wl,f
> $ llvm-dis test.bc -o -
> ; ModuleID = 'test.bc'
> target triple = "x86_64-unknown-linux-gnu"
> 
> define i32 @f() {
>  ret i32 42
> }
> 
> define i32 @main() {
>  ret i32 42
> }
> ------------------------------------------------------
> 
> Note how on the second run @f is not internalized.
> 
> Cheers,
> Rafael
If you're able to modify IR, you should also be able to keep `@f` around
by adding it to `@llvm.used`.