Joshua Gerrard via llvm-dev
2015-Nov-23 11:49 UTC
[llvm-dev] COFF::IMAGE_REL_AMD64_REL32 relocation overflow when compiling for x86_64
Some time ago I posted here regarding a relocation overflow on Windows (among other things), but the issue disappeared and so the thread got left. I've started this new thread because a) I didn't want to necro the old one and b) it felt like its own. I've now encountered the issue again and am noting down all the information I can get about it whilst it's happening. The issues is that I am getting a relocation overflow assertion inside RuntimeDyldCOFFX86_64.h inside the COFF::IMAGE_REL_AMD64_REL32 case. However, the other thread left me with the impression that I shouldn't be getting such relocation when I'm compiling for 64 bit. The only reason I can think of for this that I'm not supposed to get 32 bit relocations in the code I'm building rather than all the code being loaded. The LLVM side of the call stack looks like this: _wassert(const wchar_t * expr, const wchar_t * filename, unsigned int lineno) Line 369 C llvm::RuntimeDyldCOFFX86_64::resolveRelocation(const llvm::RelocationEntry & RE, unsigned __int64 Value) Line 81 C++ llvm::RuntimeDyldImpl::resolveRelocationList(const llvm::SmallVector<llvm::RelocationEntry,64> & Relocs, unsigned __int64 Value) Line 796 C++ llvm::RuntimeDyldImpl::resolveExternalSymbols() Line 849 C++ llvm::RuntimeDyldImpl::resolveRelocations() Line 95 C++ llvm::RuntimeDyld::resolveRelocations() Line 961 C++ llvm::orc::ObjectLinkingLayer<llvm::orc::DoNothingOnNotifyLoaded>::ConcreteLinkedObjectSet<std::shared_ptr<llvm::SectionMemoryManager>,ClangClasses::LLVMExecutionEngine::LinkingResolver * __ptr64>::Finalize() Line 112 C++ llvm::orc::ObjectLinkingLayer<llvm::orc::DoNothingOnNotifyLoaded>::findSymbolIn::__l19::<lambda>() Line 246 C++ std::_Callable_obj<unsigned __int64 <lambda>(void),0>::_ApplyX<unsigned __int64>() Line 284 C++ std::_Func_impl<std::_Callable_obj<unsigned __int64 <lambda>(void),0>,std::allocator<std::_Func_class<unsigned __int64>>,unsigned __int64>::_Do_call() Line 229 C++std::_Func_class<unsigned __int64>::operator()() Line 316 C++ llvm::orc::JITSymbol::getAddress() Line 62 C++ RelType is 4 (IMAGE_REL_AMD64_REL32). Value is 139830239098107. Addend is 0. The symbol that is currently being resolved is _fperrraise. I did some researching and it appears that this symbol resides in libcmtd.lib (for me the path is C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\lib\amd64\libcmtd.lib). The relocation type stated in that library (information gathered from dumpbin) is REL32. I'm not sure what other information there is for me to gather, could somebody please help me resolve this? Many thanks in advance! -- Joshua Gerrard JUCE Software Developer *ROLI’s **award-winning* <http://www.telegraph.co.uk/luxury/design/31520/the-seaboard-grand-piano-wins-designs-of-the-year-2014-award.html>* Seaboard GRAND, celebrated as the “**piano of the future* <http://edition.cnn.com/2013/09/27/tech/innovation/hans-zimmer-seaboard-future-piano/>*”, is now joined by the **Seaboard RISE* <https://www.youtube.com/watch?v=fGr7VbDiRNw>*, “**every bit as slimline and attractive as its bigger brother* <http://www.soundonsound.com/news?NewsID=18726>*”. The press is hailing the Seaboard RISE as “**innovative* <http://www.wired.co.uk/news/archive/2015-09/10/seaboard-rise-digital-keyboard-launch-uk-price>*”, “**expressive* <http://createdigitalmusic.com/2015/09/new-roli-instrument-wants-make-expressive-control-mainstream/>*”, “**accessible* <http://createdigitalmusic.com/2015/09/new-roli-instrument-wants-make-expressive-control-mainstream/>*”, and “**a keyboard controller that does to piano keys what 3D touch does to the iPhone* <http://www.slashgear.com/roli-seaboard-rise-is-like-3d-touch-for-musicians-11404216/>*”. Now available for preorder at **www.roli.com* <http://www.roli.com/>*.* -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20151123/72ee2dda/attachment.html>
Andy Ayers via llvm-dev
2015-Nov-23 16:27 UTC
[llvm-dev] COFF::IMAGE_REL_AMD64_REL32 relocation overflow when compiling for x86_64
Microsoft compilers have for quite a while now assumed the code you compile is going to be linked into PE images, which are limited to 4GB. So they assume a small memory model and use 32 bit relocations. If at link time it turns out your export is from a DLL the linker will insert a jump stub / dllimport into the image for you which can handle larger distances. So you can’t straightforwardly load code from a static CRT library into the Dyld-hosted process, since the latter assumes a large memory model. Your choices are: 1. Dynamically link whatever you’re compiling against the CRT (compile with /MD or /MDd as appropriate) 2. I think there has been some work on supporting small memory models in Dyld, you could try that out 3. Implement a jump stub that is “nearby” the code you’ve compiled that can branch to the target (that is, emulate what the linker does) From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of Joshua Gerrard via llvm-dev Sent: Monday, November 23, 2015 3:50 AM To: llvm-dev <llvm-dev at lists.llvm.org> Subject: [llvm-dev] COFF::IMAGE_REL_AMD64_REL32 relocation overflow when compiling for x86_64 Some time ago I posted here regarding a relocation overflow on Windows (among other things), but the issue disappeared and so the thread got left. I've started this new thread because a) I didn't want to necro the old one and b) it felt like its own. I've now encountered the issue again and am noting down all the information I can get about it whilst it's happening. The issues is that I am getting a relocation overflow assertion inside RuntimeDyldCOFFX86_64.h inside the COFF::IMAGE_REL_AMD64_REL32 case. However, the other thread left me with the impression that I shouldn't be getting such relocation when I'm compiling for 64 bit. The only reason I can think of for this that I'm not supposed to get 32 bit relocations in the code I'm building rather than all the code being loaded. The LLVM side of the call stack looks like this: _wassert(const wchar_t * expr, const wchar_t * filename, unsigned int lineno) Line 369 C llvm::RuntimeDyldCOFFX86_64::resolveRelocation(const llvm::RelocationEntry & RE, unsigned __int64 Value) Line 81 C++ llvm::RuntimeDyldImpl::resolveRelocationList(const llvm::SmallVector<llvm::RelocationEntry,64> & Relocs, unsigned __int64 Value) Line 796 C++ llvm::RuntimeDyldImpl::resolveExternalSymbols() Line 849 C++ llvm::RuntimeDyldImpl::resolveRelocations() Line 95 C++ llvm::RuntimeDyld::resolveRelocations() Line 961 C++ llvm::orc::ObjectLinkingLayer<llvm::orc::DoNothingOnNotifyLoaded>::ConcreteLinkedObjectSet<std::shared_ptr<llvm::SectionMemoryManager>,ClangClasses::LLVMExecutionEngine::LinkingResolver * __ptr64>::Finalize() Line 112 C++ llvm::orc::ObjectLinkingLayer<llvm::orc::DoNothingOnNotifyLoaded>::findSymbolIn::__l19::<lambda>() Line 246 C++ std::_Callable_obj<unsigned __int64 <lambda>(void),0>::_ApplyX<unsigned __int64>() Line 284 C++ std::_Func_impl<std::_Callable_obj<unsigned __int64 <lambda>(void),0>,std::allocator<std::_Func_class<unsigned __int64> >,unsigned __int64>::_Do_call() Line 229 C++ std::_Func_class<unsigned __int64>::operator()() Line 316 C++ llvm::orc::JITSymbol::getAddress() Line 62 C++ RelType is 4 (IMAGE_REL_AMD64_REL32). Value is 139830239098107. Addend is 0. The symbol that is currently being resolved is _fperrraise. I did some researching and it appears that this symbol resides in libcmtd.lib (for me the path is C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\lib\amd64\libcmtd.lib). The relocation type stated in that library (information gathered from dumpbin) is REL32. I'm not sure what other information there is for me to gather, could somebody please help me resolve this? Many thanks in advance! -- Joshua Gerrard JUCE Software Developer ROLI’s award-winning<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.telegraph.co.uk%2fluxury%2fdesign%2f31520%2fthe-seaboard-grand-piano-wins-designs-of-the-year-2014-award.html&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=Ykf6luCK%2f2N%2bWIfoJ2xCjeUPQcAvUo70IsWas%2boRido%3d> Seaboard GRAND, celebrated as the “piano of the future<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fedition.cnn.com%2f2013%2f09%2f27%2ftech%2finnovation%2fhans-zimmer-seaboard-future-piano%2f&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=F3ZwPreXW3KJcEwCzim4YG%2ftGYGrzT8yCS9lQ7pD4Tw%3d>”, is now joined by the Seaboard RISE<https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fwww.youtube.com%2fwatch%3fv%3dfGr7VbDiRNw&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=xaAx43eo5wz73jydxQaa3Lr6%2bvKAp1ui17tqqTELN9M%3d>, “every bit as slimline and attractive as its bigger brother<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.soundonsound.com%2fnews%3fNewsID%3d18726&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=J8FxKgDY30hPP68pFtrGMa49OeGyGOitZrPxA5BrH8U%3d>”. The press is hailing the Seaboard RISE as “innovative<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.wired.co.uk%2fnews%2farchive%2f2015-09%2f10%2fseaboard-rise-digital-keyboard-launch-uk-price&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=OKI6eDu1U0AAN4YqwQczE%2fDAxYA3i%2baOL7Vw31v6ueY%3d>”, “expressive<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fcreatedigitalmusic.com%2f2015%2f09%2fnew-roli-instrument-wants-make-expressive-control-mainstream%2f&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=iP67j7MpThqiuuKSX5cVbJDHZFKN8KHnICGWLCulVhw%3d>”, “accessible<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fcreatedigitalmusic.com%2f2015%2f09%2fnew-roli-instrument-wants-make-expressive-control-mainstream%2f&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=iP67j7MpThqiuuKSX5cVbJDHZFKN8KHnICGWLCulVhw%3d>”, and “a keyboard controller that does to piano keys what 3D touch does to the iPhone<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.slashgear.com%2froli-seaboard-rise-is-like-3d-touch-for-musicians-11404216%2f&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=P9hBMgrTEZupSsqBdX081ZpH1h%2bccISnlZ7vnBp%2bScU%3d>”. Now available for preorder at www.roli.com<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.roli.com%2f&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=%2bwHEhValdk0plr5GqFNZVsOS9yNPz9n06qH39rjF2DE%3d>. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20151123/36964956/attachment.html>
Joshua Gerrard via llvm-dev
2015-Nov-23 16:57 UTC
[llvm-dev] COFF::IMAGE_REL_AMD64_REL32 relocation overflow when compiling for x86_64
Thanks Andy, helpful as always! 1 is a possibility, but not ideal for us. Could you elaborate a little on 3? I don't really know what a jump stub is, but am guessing it's a kind of "alternative symbol" which would just act as a middle man to invoke the "real" symbol in the static library. If that's the case, I can think of a way to implement it for specific symbols, but not for the more general case. -- Joshua Gerrard JUCE Software Developer *ROLI’s **award-winning* <http://www.telegraph.co.uk/luxury/design/31520/the-seaboard-grand-piano-wins-designs-of-the-year-2014-award.html>* Seaboard GRAND, celebrated as the “**piano of the future* <http://edition.cnn.com/2013/09/27/tech/innovation/hans-zimmer-seaboard-future-piano/>*”, is now joined by the **Seaboard RISE* <https://www.youtube.com/watch?v=fGr7VbDiRNw>*, “**every bit as slimline and attractive as its bigger brother* <http://www.soundonsound.com/news?NewsID=18726>*”. The press is hailing the Seaboard RISE as “**innovative* <http://www.wired.co.uk/news/archive/2015-09/10/seaboard-rise-digital-keyboard-launch-uk-price>*”, “**expressive* <http://createdigitalmusic.com/2015/09/new-roli-instrument-wants-make-expressive-control-mainstream/>*”, “**accessible* <http://createdigitalmusic.com/2015/09/new-roli-instrument-wants-make-expressive-control-mainstream/>*”, and “**a keyboard controller that does to piano keys what 3D touch does to the iPhone* <http://www.slashgear.com/roli-seaboard-rise-is-like-3d-touch-for-musicians-11404216/>*”. Now available for preorder at **www.roli.com* <http://www.roli.com/>*.* On 23 November 2015 at 16:27, Andy Ayers <andya at microsoft.com> wrote:> Microsoft compilers have for quite a while now assumed the code you > compile is going to be linked into PE images, which are limited to 4GB. So > they assume a small memory model and use 32 bit relocations. If at link > time it turns out your export is from a DLL the linker will insert a jump > stub / dllimport into the image for you which can handle larger distances. > > > > So you can’t straightforwardly load code from a static CRT library into > the Dyld-hosted process, since the latter assumes a large memory model. > > > > Your choices are: > > > > 1. Dynamically link whatever you’re compiling against the CRT > (compile with /MD or /MDd as appropriate) > > 2. I think there has been some work on supporting small memory > models in Dyld, you could try that out > > 3. Implement a jump stub that is “nearby” the code you’ve compiled > that can branch to the target (that is, emulate what the linker does) > > > > > > *From:* llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] *On Behalf Of *Joshua > Gerrard via llvm-dev > *Sent:* Monday, November 23, 2015 3:50 AM > *To:* llvm-dev <llvm-dev at lists.llvm.org> > *Subject:* [llvm-dev] COFF::IMAGE_REL_AMD64_REL32 relocation overflow > when compiling for x86_64 > > > > Some time ago I posted here regarding a relocation overflow on Windows > (among other things), but the issue disappeared and so the thread got left. > I've started this new thread because a) I didn't want to necro the old one > and b) it felt like its own. > > I've now encountered the issue again and am noting down all the > information I can get about it whilst it's happening. > > > > The issues is that I am getting a relocation overflow assertion inside > RuntimeDyldCOFFX86_64.h inside the COFF::IMAGE_REL_AMD64_REL32 case. > > However, the other thread left me with the impression that I shouldn't be > getting such relocation when I'm compiling for 64 bit. The only reason I > can think of for this that I'm not supposed to get 32 bit relocations in > the code I'm building rather than all the code being loaded. > > > > The LLVM side of the call stack looks like this: > > > > _wassert(const wchar_t * expr, const wchar_t * filename, unsigned int > lineno) Line 369 C > > llvm::RuntimeDyldCOFFX86_64::resolveRelocation(const llvm::RelocationEntry > & RE, unsigned __int64 Value) Line 81 C++ > > llvm::RuntimeDyldImpl::resolveRelocationList(const > llvm::SmallVector<llvm::RelocationEntry,64> & Relocs, unsigned __int64 > Value) Line 796 C++ > > llvm::RuntimeDyldImpl::resolveExternalSymbols() Line 849 C++ > > llvm::RuntimeDyldImpl::resolveRelocations() Line 95 C++ > > llvm::RuntimeDyld::resolveRelocations() Line 961 C++ > > llvm::orc::ObjectLinkingLayer<llvm::orc::DoNothingOnNotifyLoaded>::ConcreteLinkedObjectSet<std::shared_ptr<llvm::SectionMemoryManager>,ClangClasses::LLVMExecutionEngine::LinkingResolver > * __ptr64>::Finalize() Line 112 C++ > > llvm::orc::ObjectLinkingLayer<llvm::orc::DoNothingOnNotifyLoaded>::findSymbolIn::__l19::<lambda>() > Line 246 C++ > > std::_Callable_obj<unsigned __int64 <lambda>(void),0>::_ApplyX<unsigned > __int64>() Line 284 C++ > > std::_Func_impl<std::_Callable_obj<unsigned __int64 > <lambda>(void),0>,std::allocator<std::_Func_class<unsigned __int64> > >,unsigned __int64>::_Do_call() Line 229 C++ > > std::_Func_class<unsigned __int64>::operator()() Line 316 C++ > > llvm::orc::JITSymbol::getAddress() Line 62 C++ > > > > RelType is 4 (IMAGE_REL_AMD64_REL32). > > Value is 139830239098107. > > Addend is 0. > > > > The symbol that is currently being resolved is _fperrraise. I did some > researching and it appears that this symbol resides in libcmtd.lib (for me > the path is C:\Program Files (x86)\Microsoft Visual Studio > 12.0\VC\lib\amd64\libcmtd.lib). > > The relocation type stated in that library (information gathered from > dumpbin) is REL32. > > > > I'm not sure what other information there is for me to gather, could > somebody please help me resolve this? > > > > Many thanks in advance! > > -- > Joshua Gerrard > JUCE Software Developer > > *ROLI’s **award-winning* > <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.telegraph.co.uk%2fluxury%2fdesign%2f31520%2fthe-seaboard-grand-piano-wins-designs-of-the-year-2014-award.html&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=Ykf6luCK%2f2N%2bWIfoJ2xCjeUPQcAvUo70IsWas%2boRido%3d>* Seaboard > GRAND, celebrated as the “**piano of the future* > <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fedition.cnn.com%2f2013%2f09%2f27%2ftech%2finnovation%2fhans-zimmer-seaboard-future-piano%2f&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=F3ZwPreXW3KJcEwCzim4YG%2ftGYGrzT8yCS9lQ7pD4Tw%3d>*”, > is now joined by the **Seaboard RISE* > <https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fwww.youtube.com%2fwatch%3fv%3dfGr7VbDiRNw&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=xaAx43eo5wz73jydxQaa3Lr6%2bvKAp1ui17tqqTELN9M%3d> > *, “**every bit as slimline and attractive as its bigger brother* > <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.soundonsound.com%2fnews%3fNewsID%3d18726&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=J8FxKgDY30hPP68pFtrGMa49OeGyGOitZrPxA5BrH8U%3d>*”. > The press is hailing the Seaboard RISE as “**innovative* > <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.wired.co.uk%2fnews%2farchive%2f2015-09%2f10%2fseaboard-rise-digital-keyboard-launch-uk-price&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=OKI6eDu1U0AAN4YqwQczE%2fDAxYA3i%2baOL7Vw31v6ueY%3d>*”, > “**expressive* > <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fcreatedigitalmusic.com%2f2015%2f09%2fnew-roli-instrument-wants-make-expressive-control-mainstream%2f&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=iP67j7MpThqiuuKSX5cVbJDHZFKN8KHnICGWLCulVhw%3d>*”, > “**accessible* > <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fcreatedigitalmusic.com%2f2015%2f09%2fnew-roli-instrument-wants-make-expressive-control-mainstream%2f&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=iP67j7MpThqiuuKSX5cVbJDHZFKN8KHnICGWLCulVhw%3d>*”, > and “**a keyboard controller that does to piano keys what 3D touch does > to the iPhone* > <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.slashgear.com%2froli-seaboard-rise-is-like-3d-touch-for-musicians-11404216%2f&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=P9hBMgrTEZupSsqBdX081ZpH1h%2bccISnlZ7vnBp%2bScU%3d>*”. > Now available for preorder at **www.roli.com* > <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.roli.com%2f&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=%2bwHEhValdk0plr5GqFNZVsOS9yNPz9n06qH39rjF2DE%3d> > *.* >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20151123/ca6e5e5e/attachment-0001.html>