Hi Lang,
Can you please let me know you think it would be right to modify
RuntimeDyldImpl::resolveExternalSymbols to allow resolvers to return 0
addresses? Something like this would be ideal for me:
void RuntimeDyldImpl::resolveExternalSymbols() {
while (!ExternalSymbolRelocations.empty()) {
StringMap<RelocationList>::iterator i =
ExternalSymbolRelocations.begin();
StringRef Name = i->first();
if (Name.size() == 0) {
// This is an absolute symbol, use an address of zero.
DEBUG(dbgs() << "Resolving absolute relocations."
<< "\n");
RelocationList &Relocs = i->second;
resolveRelocationList(Relocs, 0);
} else {
uint64_t Addr = 0;
RTDyldSymbolTable::const_iterator Loc = GlobalSymbolTable.find(Name);
if (Loc == GlobalSymbolTable.end()) {
// This is an external symbol, try to get its address from the symbol
// resolver.
Addr = Resolver.findSymbol(Name.data()).getAddress();
// The call to getSymbolAddress may have caused additional modules to
// be loaded, which may have added new entries to the
// ExternalSymbolRelocations map. Consquently, we need to update our
// iterator. This is also why retrieval of the relocation list
// associated with this symbol is deferred until below this point.
// New entries may have been added to the relocation list.
i = ExternalSymbolRelocations.find(Name);
} else {
// We found the symbol in our global table. It was probably in a
// Module that we loaded previously.
const auto &SymInfo = Loc->second;
Addr = getSectionLoadAddress(SymInfo.getSectionID()) +
SymInfo.getOffset();
}
if (Addr) {
DEBUG(dbgs() << "Resolving relocations Name: " <<
Name << "\t"
<< format("0x%lx", Addr) << "\n");
// This list may have been updated when we called getSymbolAddress, so
// don't change this code to get the list earlier.
RelocationList &Relocs = i->second;
resolveRelocationList(Relocs, Addr);
}
}
ExternalSymbolRelocations.erase(i);
}
}
Thanks,
Eugene
From: Eugene Rozenfeld
Sent: Wednesday, June 24, 2015 4:47 PM
To: 'Lang Hames'
Cc: llvmdev at cs.uiuc.edu
Subject: RE: ORC and relocations
Hi Lang,
Thank you for your reply. I tried to supply a custom Resolver that just returns
0 for the symbols I want to handle manually, but that results in a fatal error
below. If that can be changed, then yes,
it would be sufficient for my use case.
Thanks,
Eugene
void RuntimeDyldImpl::resolveExternalSymbols() {
while (!ExternalSymbolRelocations.empty()) {
StringMap<RelocationList>::iterator i =
ExternalSymbolRelocations.begin();
StringRef Name = i->first();
if (Name.size() == 0) {
// This is an absolute symbol, use an address of zero.
DEBUG(dbgs() << "Resolving absolute relocations."
<< "\n");
RelocationList &Relocs = i->second;
resolveRelocationList(Relocs, 0);
} else {
uint64_t Addr = 0;
RTDyldSymbolTable::const_iterator Loc = GlobalSymbolTable.find(Name);
if (Loc == GlobalSymbolTable.end()) {
// This is an external symbol, try to get its address from the symbol
// resolver.
Addr = Resolver.findSymbol(Name.data()).getAddress();
// The call to getSymbolAddress may have caused additional modules to
// be loaded, which may have added new entries to the
// ExternalSymbolRelocations map. Consquently, we need to update our
// iterator. This is also why retrieval of the relocation list
// associated with this symbol is deferred until below this point.
// New entries may have been added to the relocation list.
i = ExternalSymbolRelocations.find(Name);
} else {
// We found the symbol in our global table. It was probably in a
// Module that we loaded previously.
const auto &SymInfo = Loc->second;
Addr = getSectionLoadAddress(SymInfo.getSectionID()) +
SymInfo.getOffset();
}
// FIXME: Implement error handling that doesn't kill the host program!
if (!Addr) {
report_fatal_error("Program used external function '" +
Name +
"' which could not be resolved!");
}
DEBUG(dbgs() << "Resolving relocations Name: " <<
Name << "\t"
<< format("0x%lx", Addr) << "\n");
// This list may have been updated when we called getSymbolAddress, so
// don't change this code to get the list earlier.
RelocationList &Relocs = i->second;
resolveRelocationList(Relocs, Addr);
}
ExternalSymbolRelocations.erase(i);
}
}
From: Lang Hames [mailto:lhames at gmail.com]
Sent: Wednesday, June 24, 2015 4:42 PM
To: Eugene Rozenfeld
Cc: llvmdev at cs.uiuc.edu<mailto:llvmdev at cs.uiuc.edu>
Subject: Re: ORC and relocations
Hi Eugene,
There's no way to 'skip' application of a relocation.
You could use the NotifyLoadedFtor and libObject to scan the object ahead of
time and record the relocations that you'd like to override, then you could
supply a custom Resolver that just returns '0' for the symbols that you
want to handle manually.
There's no way to handle distinct relocations for the same symbol
differently: For performance reasons we only look up each symbol once and re-use
the supplied address to apply every relocation for that symbol.
Is that sufficient to support your use-case?
Cheers,
Lang.
On Wed, Jun 24, 2015 at 12:03 AM, Eugene Rozenfeld <Eugene.Rozenfeld at
microsoft.com<mailto:Eugene.Rozenfeld at microsoft.com>> wrote:
Hello,
I’m working on LLILC (a jit for the CoreCLR built on ORC), in particular, on
using LLILC as an ngen jit. I would like to have an ability to be notified of
relocations that ObjectLinkingLayer is applying and to be able to tell the
linking layer not to resolve certain relocations for external symbols (so that
the client can do some custom resolutions later). The only way I found of
looking at relocations in the client is via NotifyLoadedFtor notifications but I
couldn’t find a way of blocking a relocation resolution.
One way to achieve that is to change the Resolver::findSymbol api to allow
clients to indicate that the relocations for the symbol shouldn’t be resolved
and update RuntimeDyldImpl::resolveExternalSymbols accordingly. Would this be a
reasonable approach?
Thanks,
Eugene
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150626/d2215e28/attachment.html>
Hi Eugene, Returning null addresses to the linker is unusual enough that I'd prefer to keep it as an error. Could you return a non-zero marker value, e.g. all-ones? If not, we should add an option to RuntimeDyld like "AllowUnresolvedSymbols" and change the error condition to if '(!Addr && !AllowUnresolvedSymbols)'. Cheers, Lang. On Thu, Jun 25, 2015 at 6:35 PM, Eugene Rozenfeld < Eugene.Rozenfeld at microsoft.com> wrote:> Hi Lang, > > > > Can you please let me know you think it would be right to modify > RuntimeDyldImpl::resolveExternalSymbols to allow resolvers to return 0 > addresses? Something like this would be ideal for me: > > > > void RuntimeDyldImpl::resolveExternalSymbols() { > > while (!ExternalSymbolRelocations.empty()) { > > StringMap<RelocationList>::iterator > i = ExternalSymbolRelocations.begin(); > > > > StringRef Name = i->first(); > > if (Name.size() == 0) { > > // This is an absolute symbol, use an address of zero. > > DEBUG(dbgs() << "Resolving absolute relocations." > > << "\n"); > > RelocationList &Relocs = i->second; > > resolveRelocationList(Relocs, 0); > > } else { > > uint64_t Addr = 0; > > RTDyldSymbolTable::const_iterator > Loc = GlobalSymbolTable.find(Name); > > if (Loc == GlobalSymbolTable.end()) { > > > // This is an external symbol, try to get its address from the symbol > > // resolver. > > Addr = Resolver.findSymbol(Name.data()).getAddress(); > > > // The call to getSymbolAddress may have caused additional modules to > > // be loaded, which may have added new entries to the > > > // ExternalSymbolRelocations map. Consquently, we need to update our > > // iterator. This is also why retrieval of the relocation list > > // associated with this symbol is deferred until below this point. > > // New entries may have been added to the relocation list. > > i = ExternalSymbolRelocations.find(Name); > > } else { > > // We found the symbol in our global table. It was probably in a > > // Module that we loaded previously. > > const auto &SymInfo = Loc->second; > > Addr = getSectionLoadAddress(SymInfo.getSectionID()) + > > SymInfo.getOffset(); > > } > > > > if (Addr) { > > DEBUG(dbgs() << "Resolving relocations Name: " << Name << "\t" > > << format("0x%lx", Addr) << "\n"); > > > // This list may have been updated when we called getSymbolAddress, so > > // don't change this code to get the list earlier. > > RelocationList &Relocs = i->second; > > resolveRelocationList(Relocs, Addr); > > } > > } > > > > ExternalSymbolRelocations.erase(i); > > } > > } > > > > Thanks, > > > > Eugene > > *From:* Eugene Rozenfeld > *Sent:* Wednesday, June 24, 2015 4:47 PM > *To:* 'Lang Hames' > *Cc:* llvmdev at cs.uiuc.edu > *Subject:* RE: ORC and relocations > > > > Hi Lang, > > > > Thank you for your reply. I tried to supply a custom Resolver that just > returns 0 for the symbols I want to handle manually, but that results in a fatal > error below. If that can be changed, then yes, > > it would be sufficient for my use case. > > > > Thanks, > > > > Eugene > > > > void RuntimeDyldImpl::resolveExternalSymbols() { > > while (!ExternalSymbolRelocations.empty()) { > > StringMap<RelocationList>::iterator > i = ExternalSymbolRelocations.begin(); > > > > StringRef Name = i->first(); > > if (Name.size() == 0) { > > // This is an absolute symbol, use an address of zero. > > DEBUG(dbgs() << "Resolving absolute relocations." > > << "\n"); > > RelocationList &Relocs = i->second; > > resolveRelocationList(Relocs, 0); > > } else { > > uint64_t Addr = 0; > > RTDyldSymbolTable::const_iterator > Loc = GlobalSymbolTable.find(Name); > > if (Loc == GlobalSymbolTable.end()) { > > > // This is an external symbol, try to get its address from the symbol > > // resolver. > > Addr = Resolver.findSymbol(Name.data()).getAddress(); > > > // The call to getSymbolAddress may have caused additional modules to > > // be loaded, which may have added new entries to the > > > // ExternalSymbolRelocations map. Consquently, we need to update our > > // iterator. This is also why retrieval of the relocation list > > // associated with this symbol is deferred until below this point. > > // New entries may have been added to the relocation list. > > i = ExternalSymbolRelocations.find(Name); > > } else { > > // We found the symbol in our global table. It was probably in a > > // Module that we loaded previously. > > const auto &SymInfo = Loc->second; > > Addr = getSectionLoadAddress(SymInfo.getSectionID()) + > > SymInfo.getOffset(); > > } > > > > > // FIXME: Implement error handling that doesn't kill the host program! > > if (!Addr) { > > report_fatal_error("Program used external function '" + Name + > > "' which could not be resolved!"); > > } > > > > DEBUG(dbgs() << "Resolving relocations Name: " << Name << "\t" > > << format("0x%lx", Addr) << "\n"); > > > // This list may have been updated when we called getSymbolAddress, so > > // don't change this code to get the list earlier. > > RelocationList &Relocs = i->second; > > resolveRelocationList(Relocs, Addr); > > } > > > > ExternalSymbolRelocations.erase(i); > > } > > } > > > > > > *From:* Lang Hames [mailto:lhames at gmail.com <lhames at gmail.com>] > *Sent:* Wednesday, June 24, 2015 4:42 PM > *To:* Eugene Rozenfeld > *Cc:* llvmdev at cs.uiuc.edu > *Subject:* Re: ORC and relocations > > > > Hi Eugene, > > > > There's no way to 'skip' application of a relocation. > > > > You could use the NotifyLoadedFtor and libObject to scan the object ahead > of time and record the relocations that you'd like to override, then you > could supply a custom Resolver that just returns '0' for the symbols that > you want to handle manually. > > > > There's no way to handle distinct relocations for the same symbol > differently: For performance reasons we only look up each symbol once and > re-use the supplied address to apply every relocation for that symbol. > > > > Is that sufficient to support your use-case? > > > > Cheers, > > Lang. > > > > On Wed, Jun 24, 2015 at 12:03 AM, Eugene Rozenfeld < > Eugene.Rozenfeld at microsoft.com> wrote: > > Hello, > > > > I’m working on LLILC (a jit for the CoreCLR built on ORC), in particular, > on using LLILC as an ngen jit. I would like to have an ability to be > notified of relocations that ObjectLinkingLayer is applying and to be able > to tell the linking layer not to resolve certain relocations for external > symbols (so that the client can do some custom resolutions later). The only > way I found of looking at relocations in the client is via NotifyLoadedFtor > notifications but I couldn’t find a way of blocking a relocation resolution. > > > > One way to achieve that is to change the Resolver::findSymbol api to allow > clients to indicate that the relocations for the symbol shouldn’t be > resolved and update RuntimeDyldImpl::resolveExternalSymbols accordingly. > Would this be a reasonable approach? > > > > Thanks, > > > > Eugene > > > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150626/cee1c861/attachment.html>
Hi Lang,
Yes, I can return a non-zero marker value. Are you ok with this version?
void RuntimeDyldImpl::resolveExternalSymbols() {
while (!ExternalSymbolRelocations.empty()) {
StringMap<RelocationList>::iterator i =
ExternalSymbolRelocations.begin();
StringRef Name = i->first();
if (Name.size() == 0) {
// This is an absolute symbol, use an address of zero.
DEBUG(dbgs() << "Resolving absolute relocations."
<< "\n");
RelocationList &Relocs = i->second;
resolveRelocationList(Relocs, 0);
} else {
uint64_t Addr = 0;
RTDyldSymbolTable::const_iterator Loc = GlobalSymbolTable.find(Name);
if (Loc == GlobalSymbolTable.end()) {
// This is an external symbol, try to get its address from the symbol
// resolver.
Addr = Resolver.findSymbol(Name.data()).getAddress();
// The call to getSymbolAddress may have caused additional modules to
// be loaded, which may have added new entries to the
// ExternalSymbolRelocations map. Consquently, we need to update our
// iterator. This is also why retrieval of the relocation list
// associated with this symbol is deferred until below this point.
// New entries may have been added to the relocation list.
i = ExternalSymbolRelocations.find(Name);
} else {
// We found the symbol in our global table. It was probably in a
// Module that we loaded previously.
const auto &SymInfo = Loc->second;
Addr = getSectionLoadAddress(SymInfo.getSectionID()) +
SymInfo.getOffset();
}
// FIXME: Implement error handling that doesn't kill the host program!
if (!Addr) {
report_fatal_error("Program used external function '" +
Name +
"' which could not be resolved!");
}
// If Resolver returned UINT64_MAX, the client wants to handle this symbol
// manually and we shouldn't resolve its relocations.
if (Addr != UINT64_MAX) {
DEBUG(dbgs() << "Resolving relocations Name: " <<
Name << "\t"
<< format("0x%lx", Addr) << "\n");
// This list may have been updated when we called getSymbolAddress, so
// don't change this code to get the list earlier.
RelocationList &Relocs = i->second;
resolveRelocationList(Relocs, Addr);
}
}
ExternalSymbolRelocations.erase(i);
}
}
Thanks,
Eugene
From: Lang Hames [mailto:lhames at gmail.com]
Sent: Friday, June 26, 2015 10:28 AM
To: Eugene Rozenfeld
Cc: llvmdev at cs.uiuc.edu
Subject: Re: ORC and relocations
Hi Eugene,
Returning null addresses to the linker is unusual enough that I'd prefer to
keep it as an error.
Could you return a non-zero marker value, e.g. all-ones? If not, we should add
an option to RuntimeDyld like "AllowUnresolvedSymbols" and change the
error condition to if '(!Addr && !AllowUnresolvedSymbols)'.
Cheers,
Lang.
On Thu, Jun 25, 2015 at 6:35 PM, Eugene Rozenfeld <Eugene.Rozenfeld at
microsoft.com<mailto:Eugene.Rozenfeld at microsoft.com>> wrote:
Hi Lang,
Can you please let me know you think it would be right to modify
RuntimeDyldImpl::resolveExternalSymbols to allow resolvers to return 0
addresses? Something like this would be ideal for me:
void RuntimeDyldImpl::resolveExternalSymbols() {
while (!ExternalSymbolRelocations.empty()) {
StringMap<RelocationList>::iterator i =
ExternalSymbolRelocations.begin();
StringRef Name = i->first();
if (Name.size() == 0) {
// This is an absolute symbol, use an address of zero.
DEBUG(dbgs() << "Resolving absolute relocations."
<< "\n");
RelocationList &Relocs = i->second;
resolveRelocationList(Relocs, 0);
} else {
uint64_t Addr = 0;
RTDyldSymbolTable::const_iterator Loc = GlobalSymbolTable.find(Name);
if (Loc == GlobalSymbolTable.end()) {
// This is an external symbol, try to get its address from the symbol
// resolver.
Addr = Resolver.findSymbol(Name.data()).getAddress();
// The call to getSymbolAddress may have caused additional modules to
// be loaded, which may have added new entries to the
// ExternalSymbolRelocations map. Consquently, we need to update our
// iterator. This is also why retrieval of the relocation list
// associated with this symbol is deferred until below this point.
// New entries may have been added to the relocation list.
i = ExternalSymbolRelocations.find(Name);
} else {
// We found the symbol in our global table. It was probably in a
// Module that we loaded previously.
const auto &SymInfo = Loc->second;
Addr = getSectionLoadAddress(SymInfo.getSectionID()) +
SymInfo.getOffset();
}
if (Addr) {
DEBUG(dbgs() << "Resolving relocations Name: " <<
Name << "\t"
<< format("0x%lx", Addr) << "\n");
// This list may have been updated when we called getSymbolAddress, so
// don't change this code to get the list earlier.
RelocationList &Relocs = i->second;
resolveRelocationList(Relocs, Addr);
}
}
ExternalSymbolRelocations.erase(i);
}
}
Thanks,
Eugene
From: Eugene Rozenfeld
Sent: Wednesday, June 24, 2015 4:47 PM
To: 'Lang Hames'
Cc: llvmdev at cs.uiuc.edu<mailto:llvmdev at cs.uiuc.edu>
Subject: RE: ORC and relocations
Hi Lang,
Thank you for your reply. I tried to supply a custom Resolver that just returns
0 for the symbols I want to handle manually, but that results in a fatal error
below. If that can be changed, then yes,
it would be sufficient for my use case.
Thanks,
Eugene
From: Lang Hames [mailto:lhames at gmail.com]
Sent: Wednesday, June 24, 2015 4:42 PM
To: Eugene Rozenfeld
Cc: llvmdev at cs.uiuc.edu<mailto:llvmdev at cs.uiuc.edu>
Subject: Re: ORC and relocations
Hi Eugene,
There's no way to 'skip' application of a relocation.
You could use the NotifyLoadedFtor and libObject to scan the object ahead of
time and record the relocations that you'd like to override, then you could
supply a custom Resolver that just returns '0' for the symbols that you
want to handle manually.
There's no way to handle distinct relocations for the same symbol
differently: For performance reasons we only look up each symbol once and re-use
the supplied address to apply every relocation for that symbol.
Is that sufficient to support your use-case?
Cheers,
Lang.
On Wed, Jun 24, 2015 at 12:03 AM, Eugene Rozenfeld <Eugene.Rozenfeld at
microsoft.com<mailto:Eugene.Rozenfeld at microsoft.com>> wrote:
Hello,
I’m working on LLILC (a jit for the CoreCLR built on ORC), in particular, on
using LLILC as an ngen jit. I would like to have an ability to be notified of
relocations that ObjectLinkingLayer is applying and to be able to tell the
linking layer not to resolve certain relocations for external symbols (so that
the client can do some custom resolutions later). The only way I found of
looking at relocations in the client is via NotifyLoadedFtor notifications but I
couldn’t find a way of blocking a relocation resolution.
One way to achieve that is to change the Resolver::findSymbol api to allow
clients to indicate that the relocations for the symbol shouldn’t be resolved
and update RuntimeDyldImpl::resolveExternalSymbols accordingly. Would this be a
reasonable approach?
Thanks,
Eugene
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150630/3426bd07/attachment.html>