thr3ads.net - llvm dev - [llvm-dev] JIT, LTO and @llvm.global

If this information is useful, please help other people find it:
Share via:

Benoit Belley via llvm-dev

2017-Jun-19 16:15 UTC

[llvm-dev] JIT, LTO and @llvm.global_ctors: Looking for advise

Hi Everyone,

We are looking for advise regarding the proper use of LTO in
conjunction with just-in time generated code. Our usage scenario goes
as follows.

  1. Our front-end generates an LLVM module.

  2. A small runtime support library is linked-in. The runtime
     library is distributed as bitcode. It is generated using "clang++
     -emit-llvm' and 'llvm-link'. This allows LTO to kick-in and
     functions from the runtime support library become candidates for
     inlining. This is the desired effect that we are looking for.

  3. The resulting LLVM module is compiled and dynamically loaded. We
     are currently using the MCJIT API, but are planning to move to ORC
     very soon.

Our LLVM module linking code goes roughly as follows:

  Linker linker(jittedModule);
  std::unique_ptr<llvm::Module> moduleToLink(
    getLazyIRFileModule(bcFileName, error, context));
  linker.linkInModule(std::move(module),
                      Linker::LinkOnlyNeeded |
                      Linker::InternalizeLinkedSymbol);

Our issue is with the Linker::LinkOnlyNeeded flag. Using it has a huge
positive impact on link and compilation time :-). But, it causes the
@llvm.global_ctors and @llvm.global_dtors references from the
linked-in modules to be discarded :-(. AFAICT, the Linker code assumes
ThinLTO when the LinkOnlyNeeded flags is specified, and full-LTO
otherwise.

To resolve this, we have locally patched
llvm/lib/Linker/LinkModules.cpp with:

  bool ModuleLinker::run() {

    // ....

    if (shouldImportIntrinsicGlobalVariables()) {
      auto addIntrinsicGlobalVariable = [ValuesToLink,
                                         srcM](llvm::StringRef name) {
        if (GlobalValue *GV = SrcM->getGlobalVariable(name)) {
          ValuesToLink.insert(GV);
        }
      };

      // These are added here, because they must not be internalized.
      addIntrinsicGlobalVariable("llvm.used");
      addIntrinsicGlobalVariable("llvm.compiler.used");
      addIntrinsicGlobalVariable("llvm.global_ctors");
      addIntrinsicGlobalVariable("llvm.global_dtors");
    }

    // ...

  }

Questions:

   1. Is attempting to use llvm::Linker appropriate for our usage
      pattern ? Should we directly use llvm::IRMover instead ?

   2. Or, is our patch to ModuleLinker::run() the way to go ? Should
      we submit back a patch along these lines ?

   3. Other suggestions ?

[Note] We are currently using LLVM 4.0.1-rc2.

Cheers,
Benoit



Benoit Belley
Sr Principal Developer
M&E-Product Development Group
 
MAIN +1 514 393 1616
DIRECT +1 438 448 6304
FAX +1 514 393 0110
 
Twitter <http://twitter.com/autodesk>
Facebook <https://www.facebook.com/Autodesk>
 
Autodesk, Inc.
10 Duke Street
Montreal, Quebec, Canada H3C 2L7
www.autodesk.com <http://www.autodesk.com/>

David Blaikie via llvm-dev

2017-Jun-20 15:12 UTC

head link

[llvm-dev] JIT, LTO and @llvm.global_ctors: Looking for advise

+Lang (for JIT) & Teresa (for LTO/ThinLTO).

Sounds like maybe the LinkOnlyNeeded got reused for a bit more than the
original intent & maybe there should be more than one flag here - not sure.

On Mon, Jun 19, 2017 at 9:16 AM Benoit Belley via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi Everyone,
>
> We are looking for advise regarding the proper use of LTO in
> conjunction with just-in time generated code. Our usage scenario goes
> as follows.
>
>   1. Our front-end generates an LLVM module.
>
>   2. A small runtime support library is linked-in. The runtime
>      library is distributed as bitcode. It is generated using "clang++
>      -emit-llvm' and 'llvm-link'. This allows LTO to kick-in
and
>      functions from the runtime support library become candidates for
>      inlining. This is the desired effect that we are looking for.
>
>   3. The resulting LLVM module is compiled and dynamically loaded. We
>      are currently using the MCJIT API, but are planning to move to ORC
>      very soon.
>
> Our LLVM module linking code goes roughly as follows:
>
>   Linker linker(jittedModule);
>   std::unique_ptr<llvm::Module> moduleToLink(
>     getLazyIRFileModule(bcFileName, error, context));
>   linker.linkInModule(std::move(module),
>                       Linker::LinkOnlyNeeded |
>                       Linker::InternalizeLinkedSymbol);
>
> Our issue is with the Linker::LinkOnlyNeeded flag. Using it has a huge
> positive impact on link and compilation time :-). But, it causes the
> @llvm.global_ctors and @llvm.global_dtors references from the
> linked-in modules to be discarded :-(. AFAICT, the Linker code assumes
> ThinLTO when the LinkOnlyNeeded flags is specified, and full-LTO
> otherwise.
>
> To resolve this, we have locally patched
> llvm/lib/Linker/LinkModules.cpp with:
>
>   bool ModuleLinker::run() {
>
>     // ....
>
>     if (shouldImportIntrinsicGlobalVariables()) {
>       auto addIntrinsicGlobalVariable = [ValuesToLink,
>                                          srcM](llvm::StringRef name) {
>         if (GlobalValue *GV = SrcM->getGlobalVariable(name)) {
>           ValuesToLink.insert(GV);
>         }
>       };
>
>       // These are added here, because they must not be internalized.
>       addIntrinsicGlobalVariable("llvm.used");
>       addIntrinsicGlobalVariable("llvm.compiler.used");
>       addIntrinsicGlobalVariable("llvm.global_ctors");
>       addIntrinsicGlobalVariable("llvm.global_dtors");
>     }
>
>     // ...
>
>   }
>
> Questions:
>
>    1. Is attempting to use llvm::Linker appropriate for our usage
>       pattern ? Should we directly use llvm::IRMover instead ?
>
>    2. Or, is our patch to ModuleLinker::run() the way to go ? Should
>       we submit back a patch along these lines ?
>
>    3. Other suggestions ?
>
> [Note] We are currently using LLVM 4.0.1-rc2.
>
> Cheers,
> Benoit
>
>
>
> Benoit Belley
> Sr Principal Developer
> M&E-Product Development Group
>
> MAIN +1 514 393 1616 <(514)%20393-1616>
> DIRECT +1 438 448 6304 <(438)%20448-6304>
> FAX +1 514 393 0110 <(514)%20393-0110>
>
> Twitter <http://twitter.com/autodesk>
> Facebook <https://www.facebook.com/Autodesk>
>
> Autodesk, Inc.
> 10 Duke Street
> Montreal, Quebec, Canada H3C 2L7
> www.autodesk.com <http://www.autodesk.com/>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170620/8bc3ab44/attachment.html>

Benoit Belley via llvm-dev

2017-Jun-20 15:38 UTC

head link

[llvm-dev] JIT, LTO and @llvm.global_ctors: Looking for advise

Thanks for the hindsight.

I am currently working on a patch/potential fix which introduces a new
Linker::ImportIntrinsicGlobalVariables flag. The patch includes a unit
test reproducing the problem. Hopefully, that will help getting more
feedback. 

Note that it might take a while before I am allowed to upload the patch
since I need approval from Autodesk Legal department.

Cheers,
Benoit

Benoit Belley
Sr Principal Developer
M&E-Product Development Group
 
MAIN +1 514 393 1616
DIRECT +1 438 448 6304
FAX +1 514 393 0110
 
Twitter <http://twitter.com/autodesk>
Facebook <https://www.facebook.com/Autodesk>
 
Autodesk, Inc.
10 Duke Street
Montreal, Quebec, Canada H3C 2L7
www.autodesk.com <http://www.autodesk.com/>
 




From:  David Blaikie <dblaikie at gmail.com>
Date:  mardi 20 juin 2017 à 11:12
To:  Benoit Belley <benoit.belley at autodesk.com>, llvm-dev
<llvm-dev at lists.llvm.org>, Lang Hames <lhames at gmail.com>,
Teresa Johnson
<tejohnson at google.com>
Subject:  Re: [llvm-dev] JIT, LTO and @llvm.global_ctors: Looking for
advise

>+Lang (for JIT) & Teresa (for LTO/ThinLTO).
>
>Sounds like maybe the LinkOnlyNeeded got reused for a bit more than the
>original intent & maybe there should be more than one flag here - not
>sure.
>
>On Mon, Jun 19, 2017 at 9:16 AM Benoit Belley via llvm-dev
><llvm-dev at lists.llvm.org> wrote:
>
>
>Hi Everyone,
>
>We are looking for advise regarding the proper use of LTO in
>conjunction with just-in time generated code. Our usage scenario goes
>as follows.
>
>  1. Our front-end generates an LLVM module.
>
>  2. A small runtime support library is linked-in. The runtime
>     library is distributed as bitcode. It is generated using "clang++
>     -emit-llvm' and 'llvm-link'. This allows LTO to kick-in and
>     functions from the runtime support library become candidates for
>     inlining. This is the desired effect that we are looking for.
>
>  3. The resulting LLVM module is compiled and dynamically loaded. We
>     are currently using the MCJIT API, but are planning to move to ORC
>     very soon.
>
>Our LLVM module linking code goes roughly as follows:
>
>  Linker linker(jittedModule);
>  std::unique_ptr<llvm::Module> moduleToLink(
>    getLazyIRFileModule(bcFileName, error, context));
>  linker.linkInModule(std::move(module),
>                      Linker::LinkOnlyNeeded |
>                      Linker::InternalizeLinkedSymbol);
>
>Our issue is with the Linker::LinkOnlyNeeded flag. Using it has a huge
>positive impact on link and compilation time :-). But, it causes the
>@llvm.global_ctors and @llvm.global_dtors references from the
>linked-in modules to be discarded :-(. AFAICT, the Linker code assumes
>ThinLTO when the LinkOnlyNeeded flags is specified, and full-LTO
>otherwise.
>
>To resolve this, we have locally patched
>llvm/lib/Linker/LinkModules.cpp with:
>
>  bool ModuleLinker::run() {
>
>    // ....
>
>    if (shouldImportIntrinsicGlobalVariables()) {
>      auto addIntrinsicGlobalVariable = [ValuesToLink,
>                                         srcM](llvm::StringRef name) {
>        if (GlobalValue *GV = SrcM->getGlobalVariable(name)) {
>          ValuesToLink.insert(GV);
>        }
>      };
>
>      // These are added here, because they must not be internalized.
>      addIntrinsicGlobalVariable("llvm.used");
>      addIntrinsicGlobalVariable("llvm.compiler.used");
>      addIntrinsicGlobalVariable("llvm.global_ctors");
>      addIntrinsicGlobalVariable("llvm.global_dtors");
>    }
>
>    // ...
>
>  }
>
>Questions:
>
>   1. Is attempting to use llvm::Linker appropriate for our usage
>      pattern ? Should we directly use llvm::IRMover instead ?
>
>   2. Or, is our patch to ModuleLinker::run() the way to go ? Should
>      we submit back a patch along these lines ?
>
>   3. Other suggestions ?
>
>[Note] We are currently using LLVM 4.0.1-rc2.
>
>Cheers,
>Benoit
>
>
>
>Benoit Belley
>Sr Principal Developer
>M&E-Product Development Group
>
>MAIN +1 514 393 1616 <tel:(514)%20393-1616>
>DIRECT +1 438 448 6304 <tel:(438)%20448-6304>
>FAX +1 514 393 0110 <tel:(514)%20393-0110>
>
>Twitter <http://twitter.com/autodesk>
>Facebook <https://www.facebook.com/Autodesk>
>
>Autodesk, Inc.
>10 Duke Street
>Montreal, Quebec, Canada H3C 2L7
>www.autodesk.com <http://www.autodesk.com>
<http://www.autodesk.com/>
>
>
>_______________________________________________
>LLVM Developers mailing list
>llvm-dev at lists.llvm.org
>http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Possibly Parallel Threads

Search for more apparently analagous threads

llvm dev - Jun 2017 - JIT, LTO and @llvm.global_ctors: Looking for advise

[llvm-dev] JIT, LTO and @llvm.global_ctors: Looking for advise

[llvm-dev] JIT, LTO and @llvm.global_ctors: Looking for advise

[llvm-dev] JIT, LTO and @llvm.global_ctors: Looking for advise

Possibly Parallel Threads