llvm dev - Feb 2021 - saving and restoring multiple modules for Orc JIT

Well, I have worked around this problem by implementing my own little file
format to hold the simple concatenation of any number of bitcode files. It
does not seem like this should be necessary, though. If anyone comes up
with a better solution in the future, I am interested...

Geoff

On Tue, Feb 2, 2021 at 12:26 PM Geoff Levner <glevner at gmail.com> wrote:
> Greetings, LLVM developers.
>
> I am trying to save the bitcode for multiple modules so that I can reload
> them later and add them to my Orc-based JIT.
>
> If I save each module to its own file, I have no problem. But I would like
> to save them to a single file. When I do that, I sometimes get duplicate
> symbol definitions when I reload the modules:
>
>     Duplicate definition of symbol
'$./path/to/my/file.bc.__inits.0'
>
> It looks to me as though this is a problem with static constructors in the
> modules, as though each module puts its constructors in a symbol with the
> same name...?
>
> Does anybody have a better idea than I am of what is going on, or how to
> solve this?
>
> I am using the 11.x release branch on Linux, FWIW. To reload the modules,
> I use a call to getBitcodeModuleList(), followed by calls to
> BitcodeModule::parseModule(), and finally calls to
> orc::LLJIT::addIRModule().
>
> Geoff
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210205/8dfa93e6/attachment.html>

Hi Geoff,

If I save each module to its own file, I have no problem. But I would
like
> to save them to a single file. When I do that, I sometimes get duplicate
> symbol definitions when I reload the modules:
>     Duplicate definition of symbol
'$./path/to/my/file.bc.__inits.0'

That's a pseudo-symbol representing the static initializers. Orc
auto-generates it based on the module identifier, and the module identifier
defaults to the path-name of the module file (I don't recall whether this
happens when the module is stored or loaded). I think you could fix this by
setting the module identifier for each module to a unique string (via
Module::setModuleIdentifier) before adding it to the JIT.

I'll see if I can add de-duping logic to the JIT API side too.

-- Lang.

On Sat, Feb 6, 2021 at 12:32 AM Geoff Levner <glevner at gmail.com> wrote:
> Well, I have worked around this problem by implementing my own little file
> format to hold the simple concatenation of any number of bitcode files. It
> does not seem like this should be necessary, though. If anyone comes up
> with a better solution in the future, I am interested...
>
> Geoff
>
> On Tue, Feb 2, 2021 at 12:26 PM Geoff Levner <glevner at gmail.com>
wrote:
>
>> Greetings, LLVM developers.
>>
>> I am trying to save the bitcode for multiple modules so that I can
reload
>> them later and add them to my Orc-based JIT.
>>
>> If I save each module to its own file, I have no problem. But I would
>> like to save them to a single file. When I do that, I sometimes get
>> duplicate symbol definitions when I reload the modules:
>>
>>     Duplicate definition of symbol
'$./path/to/my/file.bc.__inits.0'
>>
>> It looks to me as though this is a problem with static constructors in
>> the modules, as though each module puts its constructors in a symbol
with
>> the same name...?
>>
>> Does anybody have a better idea than I am of what is going on, or how
to
>> solve this?
>>
>> I am using the 11.x release branch on Linux, FWIW. To reload the
modules,
>> I use a call to getBitcodeModuleList(), followed by calls to
>> BitcodeModule::parseModule(), and finally calls to
>> orc::LLJIT::addIRModule().
>>
>> Geoff
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210212/e2e07c2f/attachment.html>