Yes, I’m handling all internal and external relocations manually in
NotifyLoadedFtor and I already verified that I get the behavior I need if I
comment out the call to resolveRelocations.
I would like to reuse ObjectLinkingLayer::addObjectSet (which eventually calls
RuntimeDyld::loadObject), which has the right calls to the memory manager and
also RuntimeDyld::registerEHFrames.
I understand that resolveRelocations is normally the main job of
ObjectLinkingLayer and as I said I can create my own ObjectLinkingLayer or
something instead of it that has these calls but it doesn’t feel right if all I
need to reuse ObjectLinkingLayer is to avoid the resolveRelocations call.
Eugene
From: Lang Hames [mailto:lhames at gmail.com]
Sent: Wednesday, July 22, 2015 8:17 PM
To: Eugene Rozenfeld <Eugene.Rozenfeld at microsoft.com>
Cc: llvmdev at cs.uiuc.edu
Subject: Re: ORC and relocations
Hi Eugene,
Skipping the call to resolveRelocations would disable many (if not all) internal
relocations too. Is that the desired behavior?
At that point there's not much left for RuntimeDyld (or the
ObjectLinkingLayer) to do. Would something like a NoopLinkingLayer be a workable
solution?
Cheers,
Lang.
On Wed, Jul 22, 2015 at 7:26 PM, Eugene Rozenfeld <Eugene.Rozenfeld at
microsoft.com<mailto:Eugene.Rozenfeld at microsoft.com>> wrote:
Hi Lang,
It turns out I also need an ability to tell the object linking layer not to
apply any relocations. I need to skip this step below.
The only way I can see I can achieve that is by creating my own
ObjectLinkingLayer that would duplicate almost all of orc::ObjectLinkingLayer.
I’d like to avoid that. An alternative it to pass a flag to
orc::ObjectLinkingLayer constructor and
orc::ObjectLinkingLayer::ConcreteLinkedObjectSet constructor
to indicate whether relocation resolution should be performed. Would you be ok
with such a change?
Thanks,
Eugene
template <typename NotifyLoadedFtor = DoNothingOnNotifyLoaded>
class ObjectLinkingLayer : public ObjectLinkingLayerBase {
private:
template <typename MemoryManagerPtrT, typename SymbolResolverPtrT>
class ConcreteLinkedObjectSet : public LinkedObjectSet {
public:
ConcreteLinkedObjectSet(MemoryManagerPtrT MemMgr,
SymbolResolverPtrT Resolver)
: LinkedObjectSet(*MemMgr, *Resolver), MemMgr(std::move(MemMgr)),
Resolver(std::move(Resolver)) { }
void Finalize() override {
State = Finalizing;
RTDyld->resolveRelocations();
RTDyld->registerEHFrames();
MemMgr->finalizeMemory();
OwnedBuffers.clear();
State = Finalized;
}
From: Lang Hames [mailto:lhames at gmail.com<mailto:lhames at gmail.com>]
Sent: Friday, July 3, 2015 6:36 PM
To: Eugene Rozenfeld <Eugene.Rozenfeld at
microsoft.com<mailto:Eugene.Rozenfeld at microsoft.com>>
Cc: llvmdev at cs.uiuc.edu<mailto:llvmdev at cs.uiuc.edu>
Subject: Re: ORC and relocations
Hi Eugene,
Sorry for the delayed reply. This looks good to me - I've applied it (along
with some extra code to make it testable) in r241383.
Cheers,
Lang.
On Mon, Jun 29, 2015 at 7:31 PM, Eugene Rozenfeld <Eugene.Rozenfeld at
microsoft.com<mailto:Eugene.Rozenfeld at microsoft.com>> wrote:
Hi Lang,
Yes, I can return a non-zero marker value. Are you ok with this version?
void RuntimeDyldImpl::resolveExternalSymbols() {
while (!ExternalSymbolRelocations.empty()) {
StringMap<RelocationList>::iterator i =
ExternalSymbolRelocations.begin();
StringRef Name = i->first();
if (Name.size() == 0) {
// This is an absolute symbol, use an address of zero.
DEBUG(dbgs() << "Resolving absolute relocations."
<< "\n");
RelocationList &Relocs = i->second;
resolveRelocationList(Relocs, 0);
} else {
uint64_t Addr = 0;
RTDyldSymbolTable::const_iterator Loc = GlobalSymbolTable.find(Name);
if (Loc == GlobalSymbolTable.end()) {
// This is an external symbol, try to get its address from the symbol
// resolver.
Addr = Resolver.findSymbol(Name.data()).getAddress();
// The call to getSymbolAddress may have caused additional modules to
// be loaded, which may have added new entries to the
// ExternalSymbolRelocations map. Consquently, we need to update our
// iterator. This is also why retrieval of the relocation list
// associated with this symbol is deferred until below this point.
// New entries may have been added to the relocation list.
i = ExternalSymbolRelocations.find(Name);
} else {
// We found the symbol in our global table. It was probably in a
// Module that we loaded previously.
const auto &SymInfo = Loc->second;
Addr = getSectionLoadAddress(SymInfo.getSectionID()) +
SymInfo.getOffset();
}
// FIXME: Implement error handling that doesn't kill the host program!
if (!Addr) {
report_fatal_error("Program used external function '" +
Name +
"' which could not be resolved!");
}
// If Resolver returned UINT64_MAX, the client wants to handle this symbol
// manually and we shouldn't resolve its relocations.
if (Addr != UINT64_MAX) {
DEBUG(dbgs() << "Resolving relocations Name: " <<
Name << "\t"
<< format("0x%lx", Addr) << "\n");
// This list may have been updated when we called getSymbolAddress, so
// don't change this code to get the list earlier.
RelocationList &Relocs = i->second;
resolveRelocationList(Relocs, Addr);
}
}
ExternalSymbolRelocations.erase(i);
}
}
Thanks,
Eugene
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150723/a0ba7517/attachment.html>
Hi Eugene, Sorry for the delayed reply. Custom relocations weren't something I had in mind when I designed Orc, so they raise some interesting design questions which I don't have good answers to yet. (E.g. The interface for the Orc layer concept assumes that there's a RuntimeDyld instance embedded at the bottom of the stack. That's why addModuleSet takes a MemoryManager and SymbolResolver. If there's no RuntimeDyld instance at the bottom of the stack, it's not obvious that the interface should require those). For now I recommend just creating a local copy of the ObjectLinkingLayer and removing the call to resolveRelocations. I'll keep thinking about the best way to support this use case going forward. Cheers, Lang. On Wed, Jul 22, 2015 at 11:51 PM, Eugene Rozenfeld < Eugene.Rozenfeld at microsoft.com> wrote:> Yes, I’m handling all internal and external relocations manually in > NotifyLoadedFtor and I already verified that I get the behavior I need if I > comment out the call to resolveRelocations. > > > > I would like to reuse ObjectLinkingLayer::addObjectSet (which eventually > calls RuntimeDyld::loadObject), which has the right calls to the memory > manager and also RuntimeDyld::registerEHFrames. > > > > I understand that resolveRelocations is normally the main job of > ObjectLinkingLayer and as I said I can create my own ObjectLinkingLayer or > something instead of it that has these calls but it doesn’t feel right if > all I need to reuse ObjectLinkingLayer is to avoid the resolveRelocations > call. > > > > Eugene > > > > *From:* Lang Hames [mailto:lhames at gmail.com] > *Sent:* Wednesday, July 22, 2015 8:17 PM > > *To:* Eugene Rozenfeld <Eugene.Rozenfeld at microsoft.com> > *Cc:* llvmdev at cs.uiuc.edu > *Subject:* Re: ORC and relocations > > > > Hi Eugene, > > > > Skipping the call to resolveRelocations would disable many (if not all) > internal relocations too. Is that the desired behavior? > > > > At that point there's not much left for RuntimeDyld (or the > ObjectLinkingLayer) to do. Would something like a NoopLinkingLayer be a > workable solution? > > > > Cheers, > > Lang. > > > > > > On Wed, Jul 22, 2015 at 7:26 PM, Eugene Rozenfeld < > Eugene.Rozenfeld at microsoft.com> wrote: > > Hi Lang, > > > > It turns out I also need an ability to tell the object linking layer not > to apply any relocations. I need to skip this step below. > > The only way I can see I can achieve that is by creating my own > ObjectLinkingLayer that would duplicate almost all of > orc::ObjectLinkingLayer. > > I’d like to avoid that. An alternative it to pass a flag to > orc::ObjectLinkingLayer constructor and > orc::ObjectLinkingLayer::ConcreteLinkedObjectSet constructor > > to indicate whether relocation resolution should be performed. Would you > be ok with such a change? > > > > Thanks, > > > > Eugene > > > > template <typename NotifyLoadedFtor = DoNothingOnNotifyLoaded> > > class ObjectLinkingLayer : public ObjectLinkingLayerBase { > > private: > > > > template <typename MemoryManagerPtrT, typename SymbolResolverPtrT> > > class ConcreteLinkedObjectSet : public LinkedObjectSet { > > public: > > ConcreteLinkedObjectSet(MemoryManagerPtrT MemMgr, > > SymbolResolverPtrT Resolver) > > : LinkedObjectSet(*MemMgr, *Resolver), MemMgr(std::move(MemMgr)), > > Resolver(std::move(Resolver)) { } > > > > void Finalize() override { > > State = Finalizing; > > RTDyld->resolveRelocations(); > > RTDyld->registerEHFrames(); > > MemMgr->finalizeMemory(); > > OwnedBuffers.clear(); > > State = Finalized; > > } > > > > > > *From:* Lang Hames [mailto:lhames at gmail.com] > *Sent:* Friday, July 3, 2015 6:36 PM > *To:* Eugene Rozenfeld <Eugene.Rozenfeld at microsoft.com> > *Cc:* llvmdev at cs.uiuc.edu > *Subject:* Re: ORC and relocations > > > > Hi Eugene, > > > > Sorry for the delayed reply. This looks good to me - I've applied it > (along with some extra code to make it testable) in r241383. > > > > Cheers, > > Lang. > > > > On Mon, Jun 29, 2015 at 7:31 PM, Eugene Rozenfeld < > Eugene.Rozenfeld at microsoft.com> wrote: > > Hi Lang, > > > > Yes, I can return a non-zero marker value. Are you ok with this version? > > > > void RuntimeDyldImpl::resolveExternalSymbols() { > > while (!ExternalSymbolRelocations.empty()) { > > StringMap<RelocationList>::iterator > i = ExternalSymbolRelocations.begin(); > > > > StringRef Name = i->first(); > > if (Name.size() == 0) { > > // This is an absolute symbol, use an address of zero. > > DEBUG(dbgs() << "Resolving absolute relocations." > > << "\n"); > > RelocationList &Relocs = i->second; > > resolveRelocationList(Relocs, 0); > > } else { > > uint64_t Addr = 0; > > RTDyldSymbolTable::const_iterator > Loc = GlobalSymbolTable.find(Name); > > if (Loc == GlobalSymbolTable.end()) { > > > // This is an external symbol, try to get its address from the symbol > > // resolver. > > Addr = Resolver.findSymbol(Name.data()).getAddress(); > > > // The call to getSymbolAddress may have caused additional modules to > > // be loaded, which may have added new entries to the > > > // ExternalSymbolRelocations map. Consquently, we need to update our > > // iterator. This is also why retrieval of the relocation list > > // associated with this symbol is deferred until below this point. > > // New entries may have been added to the relocation list. > > i = ExternalSymbolRelocations.find(Name); > > } else { > > // We found the symbol in our global table. It was probably in a > > // Module that we loaded previously. > > const auto &SymInfo = Loc->second; > > Addr = getSectionLoadAddress(SymInfo.getSectionID()) + > > SymInfo.getOffset(); > > } > > > > > // FIXME: Implement error handling that doesn't kill the host program! > > if (!Addr) { > > report_fatal_error("Program used external function '" + Name + > > "' which could not be resolved!"); > > } > > > > // If Resolver returned UINT64_MAX, the client wants to handle this symbol > > // manually and we shouldn't resolve its relocations. > > if (Addr != UINT64_MAX) { > > DEBUG(dbgs() << "Resolving relocations Name: " << Name << "\t" > > << format("0x%lx", Addr) << "\n"); > > > // This list may have been updated when we called getSymbolAddress, so > > // don't change this code to get the list earlier. > > RelocationList &Relocs = i->second; > > resolveRelocationList(Relocs, Addr); > > } > > } > > > > ExternalSymbolRelocations.erase(i); > > } > > } > > > > > > Thanks, > > > > Eugene > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150724/a06ff0cf/attachment.html>
Hi Lang,
Sounds good, I’ll create a local copy of ObjectLinkingLayer. Hopefully it won’t
be necessary in the future.
Thanks,
Eugene
From: Lang Hames [mailto:lhames at gmail.com]
Sent: Friday, July 24, 2015 1:24 PM
To: Eugene Rozenfeld <Eugene.Rozenfeld at microsoft.com>
Cc: llvmdev at cs.uiuc.edu
Subject: Re: ORC and relocations
Hi Eugene,
Sorry for the delayed reply. Custom relocations weren't something I had in
mind when I designed Orc, so they raise some interesting design questions which
I don't have good answers to yet. (E.g. The interface for the Orc layer
concept assumes that there's a RuntimeDyld instance embedded at the bottom
of the stack. That's why addModuleSet takes a MemoryManager and
SymbolResolver. If there's no RuntimeDyld instance at the bottom of the
stack, it's not obvious that the interface should require those).
For now I recommend just creating a local copy of the ObjectLinkingLayer and
removing the call to resolveRelocations. I'll keep thinking about the best
way to support this use case going forward.
Cheers,
Lang.
On Wed, Jul 22, 2015 at 11:51 PM, Eugene Rozenfeld <Eugene.Rozenfeld at
microsoft.com<mailto:Eugene.Rozenfeld at microsoft.com>> wrote:
Yes, I’m handling all internal and external relocations manually in
NotifyLoadedFtor and I already verified that I get the behavior I need if I
comment out the call to resolveRelocations.
I would like to reuse ObjectLinkingLayer::addObjectSet (which eventually calls
RuntimeDyld::loadObject), which has the right calls to the memory manager and
also RuntimeDyld::registerEHFrames.
I understand that resolveRelocations is normally the main job of
ObjectLinkingLayer and as I said I can create my own ObjectLinkingLayer or
something instead of it that has these calls but it doesn’t feel right if all I
need to reuse ObjectLinkingLayer is to avoid the resolveRelocations call.
Eugene
From: Lang Hames [mailto:lhames at gmail.com<mailto:lhames at gmail.com>]
Sent: Wednesday, July 22, 2015 8:17 PM
To: Eugene Rozenfeld <Eugene.Rozenfeld at
microsoft.com<mailto:Eugene.Rozenfeld at microsoft.com>>
Cc: llvmdev at cs.uiuc.edu<mailto:llvmdev at cs.uiuc.edu>
Subject: Re: ORC and relocations
Hi Eugene,
Skipping the call to resolveRelocations would disable many (if not all) internal
relocations too. Is that the desired behavior?
At that point there's not much left for RuntimeDyld (or the
ObjectLinkingLayer) to do. Would something like a NoopLinkingLayer be a workable
solution?
Cheers,
Lang.
On Wed, Jul 22, 2015 at 7:26 PM, Eugene Rozenfeld <Eugene.Rozenfeld at
microsoft.com<mailto:Eugene.Rozenfeld at microsoft.com>> wrote:
Hi Lang,
It turns out I also need an ability to tell the object linking layer not to
apply any relocations. I need to skip this step below.
The only way I can see I can achieve that is by creating my own
ObjectLinkingLayer that would duplicate almost all of orc::ObjectLinkingLayer.
I’d like to avoid that. An alternative it to pass a flag to
orc::ObjectLinkingLayer constructor and
orc::ObjectLinkingLayer::ConcreteLinkedObjectSet constructor
to indicate whether relocation resolution should be performed. Would you be ok
with such a change?
Thanks,
Eugene
template <typename NotifyLoadedFtor = DoNothingOnNotifyLoaded>
class ObjectLinkingLayer : public ObjectLinkingLayerBase {
private:
template <typename MemoryManagerPtrT, typename SymbolResolverPtrT>
class ConcreteLinkedObjectSet : public LinkedObjectSet {
public:
ConcreteLinkedObjectSet(MemoryManagerPtrT MemMgr,
SymbolResolverPtrT Resolver)
: LinkedObjectSet(*MemMgr, *Resolver), MemMgr(std::move(MemMgr)),
Resolver(std::move(Resolver)) { }
void Finalize() override {
State = Finalizing;
RTDyld->resolveRelocations();
RTDyld->registerEHFrames();
MemMgr->finalizeMemory();
OwnedBuffers.clear();
State = Finalized;
}
From: Lang Hames [mailto:lhames at gmail.com<mailto:lhames at gmail.com>]
Sent: Friday, July 3, 2015 6:36 PM
To: Eugene Rozenfeld <Eugene.Rozenfeld at
microsoft.com<mailto:Eugene.Rozenfeld at microsoft.com>>
Cc: llvmdev at cs.uiuc.edu<mailto:llvmdev at cs.uiuc.edu>
Subject: Re: ORC and relocations
Hi Eugene,
Sorry for the delayed reply. This looks good to me - I've applied it (along
with some extra code to make it testable) in r241383.
Cheers,
Lang.
On Mon, Jun 29, 2015 at 7:31 PM, Eugene Rozenfeld <Eugene.Rozenfeld at
microsoft.com<mailto:Eugene.Rozenfeld at microsoft.com>> wrote:
Hi Lang,
Yes, I can return a non-zero marker value. Are you ok with this version?
void RuntimeDyldImpl::resolveExternalSymbols() {
while (!ExternalSymbolRelocations.empty()) {
StringMap<RelocationList>::iterator i =
ExternalSymbolRelocations.begin();
StringRef Name = i->first();
if (Name.size() == 0) {
// This is an absolute symbol, use an address of zero.
DEBUG(dbgs() << "Resolving absolute relocations."
<< "\n");
RelocationList &Relocs = i->second;
resolveRelocationList(Relocs, 0);
} else {
uint64_t Addr = 0;
RTDyldSymbolTable::const_iterator Loc = GlobalSymbolTable.find(Name);
if (Loc == GlobalSymbolTable.end()) {
// This is an external symbol, try to get its address from the symbol
// resolver.
Addr = Resolver.findSymbol(Name.data()).getAddress();
// The call to getSymbolAddress may have caused additional modules to
// be loaded, which may have added new entries to the
// ExternalSymbolRelocations map. Consquently, we need to update our
// iterator. This is also why retrieval of the relocation list
// associated with this symbol is deferred until below this point.
// New entries may have been added to the relocation list.
i = ExternalSymbolRelocations.find(Name);
} else {
// We found the symbol in our global table. It was probably in a
// Module that we loaded previously.
const auto &SymInfo = Loc->second;
Addr = getSectionLoadAddress(SymInfo.getSectionID()) +
SymInfo.getOffset();
}
// FIXME: Implement error handling that doesn't kill the host program!
if (!Addr) {
report_fatal_error("Program used external function '" +
Name +
"' which could not be resolved!");
}
// If Resolver returned UINT64_MAX, the client wants to handle this symbol
// manually and we shouldn't resolve its relocations.
if (Addr != UINT64_MAX) {
DEBUG(dbgs() << "Resolving relocations Name: " <<
Name << "\t"
<< format("0x%lx", Addr) << "\n");
// This list may have been updated when we called getSymbolAddress, so
// don't change this code to get the list earlier.
RelocationList &Relocs = i->second;
resolveRelocationList(Relocs, Addr);
}
}
ExternalSymbolRelocations.erase(i);
}
}
Thanks,
Eugene
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150725/2667223b/attachment.html>