llvm dev - Jun 2015 - [LLVMdev] Linking modules across contexts crashes

> On 2015-Jun-02, at 12:37, Yuri <yuri at rawbw.com> wrote:
> 
> On 06/01/2015 11:43, Duncan P. N. Exon Smith wrote:
>> You can round-trip to bitcode, reading the module into the
>> destination context.  The following pseudo-code gives the idea:
>> 
>>     bool linkModuleFromDifferentContext(Module &D, const Module
&S) {
>>       SmallVector<char, 256> Buffer;
>>       writeBitcodeToBuffer(S, Buffer);
>> 
>>       std::unique_ptr<Module> M =
readBitcodeFromBuffer(D.getContext());
>>       return Linker::LinkModules(&D, M.get());
>>     }
> 
> Duncan,
> 
> Thanks for this workaround, i works. However, going to binary and back
causes the significant bump in the process user time (14s->16s), while the
wall clock time still lower due to parallelization.
> 
> There should be the function llvm::MergeContexts, similar to
llvm::Linker::LinkModules. It should merge contexts as containers, with an error
when this isn't possible, without changing most of the objects at all. It
should cause the source context to disappear.
If you want to work on a solution, I think a better approach would be
adding API for one (or both) of:

    class Module {
    public:
        /// Move this module to the given context.
        void changeContext(LLVMContext &NewContext);
    };

    /// Remap the given module in a different context.
    std::unique_ptr<Module> remapModuleToContext(
        const Module &M, LLVMContext &NewContext);

I'm not sure what your exact use case is, but I could see either/both
interfaces being useful.  For example, you could change
`Linker::LinkModules()` to something like:

    bool Linker::LinkModules(Module &Dest, Module &Src) {
      if (&Src.getContext() != &Dest.getContext())
        // Recreate Src in Dest's context and link that.
        return LinkModules(
                   Dest,
                   remapModuleToContext(Src, Dest.getContext()).get());

      // Old code...
    }

or:

    bool Linker::LinkModules(Module &Dest, Module &Src) {
      if (&Src.getContext() != &Dest.getContext())
        // Destructively move Src to Dest's context.
        Src.changeContext(Dest.getContext());

      // Old code...
    }

Patches welcome!

On 06/02/2015 13:05, Duncan P. N. Exon Smith wrote:
> If you want to work on a solution, I think a better approach would be
> adding API for one (or both) of:
>
>      class Module {
>      public:
>          /// Move this module to the given context.
>          void changeContext(LLVMContext &NewContext);
>      };
>
>      /// Remap the given module in a different context.
>      std::unique_ptr<Module> remapModuleToContext(
>          const Module &M, LLVMContext &NewContext);
While working on the patch, I have two questions:

1. How to find all constants in Module? Does this code find all of them, 
or they are somewhere else too?
   for (GlobalVariable &GV : globals()) {
     if (auto C = static_cast<Constant*>(GV.Op<0>().get())) {
       ... C is Constant*
     }
   }

2. How to create an equivalent ConstantArray/ConstantStruct objects in 
another context? ConstantArray::get(ArrayType *T, ArrayRef<Constant*>V) 
creates it, and puts it into LLVMContextImpl::ArrayConstants. 
ArrayConstants has the DenseMap ConstantUniqueMap::Map with 
ConstantArray* as a key, but how to read back ArrayRef<Constant*> 
ConstantArray was created with?

Yuri

> 1. How to find all constants in Module? Does this code find all of them, or
> they are somewhere else too?
>   for (GlobalVariable &GV : globals()) {
>     if (auto C = static_cast<Constant*>(GV.Op<0>().get())) {
>       ... C is Constant*
>     }
>   }
Constants are unfortunately part of the Context, not the module :-(

Cheers,
Rafael