Ivan Serdyuk via llvm-dev
2020-Aug-16 23:17 UTC
[llvm-dev] Supporting libunwind on Windows 10 (32bit; 64bit) for MSVC and Clang
Martin, good to hear from you. Thanks for the effort - I will test on 32bit and 64bit Windows 10. I will report ASAP. Ivan On Sun, Aug 16, 2020 at 8:42 PM Martin Storsjö <martin at martin.st> wrote:> On Sat, 15 Aug 2020, Ivan Serdyuk wrote: > > > > > > > On Sat, Aug 15, 2020 at 8:39 PM Martin Storsjö <martin at martin.st> wrote: > > Hi, > > > > > > On Sat, 15 Aug 2020, Ivan Serdyuk wrote: > > > > > Just as Shoaib said, libunwind only is useful in > > environments > > > that use > > > the Itanium C++ ABI - there's really no use for it in an > > MSVC > > > context > > > (either using MSVC or clang-cl to compile it). > > > > > > The particular linker error comes from the fact that > > there's > > > functions > > > implemented in assembly, that expect the function name > > to be > > > mangled the > > > itanium way, while the object files built by the > > compiler expect > > > the > > > symbols to use the MSVC C++ name mangling, so there's an > > > undefined > > > reference. > > > > > > > > > I see, thanks. > > > > > > Which options exist for MSVC, in sense an alternative to > > libunwind and > > > libbacktrace ? > > > > Well for libunwind, there's really no use for it in an MSVC > > setting. All > > the unwinding functionality is already built into the operating > > system, > > available via the Rtl*Unwind* functions. > > > > > > Martin, > > you mean these functions > > > https://docs.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtlunwind > > > > > https://docs.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtlunwind > > 2 > > > https://docs.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtlunwind > > ex > > > https://docs.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtlvirtua > > lunwind > > ? > > Yes, those are the functions used for unwinding on windows. > > > For libbacktrace, I guess the _Unwind_Backtrace function in > > libunwind > > might work and be useful, even if the rest of libunwind doesn't > > make sense > > in such a context. > > Just FWIW, I made a RFC patch that fixes building libunwind in MSVC mode > (still requiring clang-cl though), at https://reviews.llvm.org/D86041 - > mostly just to show what it takes - not convinced that it necessarily is > something that should be done. > > It does seem like _Unwind_Backtrace in libunwind doesn't work properly on > x86_64 windows at the moment though, but on aarch64 it seems to do what > one would expect. > > // Martin >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200816/9d31490c/attachment.html>
Ivan Serdyuk via llvm-dev
2020-Aug-16 23:30 UTC
[llvm-dev] Supporting libunwind on Windows 10 (32bit; 64bit) for MSVC and Clang
This is what Ken Johnson replied: On Sun, Aug 16, 2020 at 4:09 PM Skywing <Skywing at valhallalegends.com> wrote:> I just briefly skimmed the discussion, so hopefully I didn't misunderstand > what you are trying to do ... but if your objective is to capture a stack > trace under NT, the OS exports several services for this purpose, depending > on what you need out of such a stack trace : > > > > 1) There is a "pre-cooked" stack trace facility that will perform the > entire stack trace for you (optionally skipping some frames at the top), > returning an array of program counter values for each return address on the > stack: > https://docs.microsoft.com/en-us/previous-versions/windows/desktop/legacy/bb204633(v=vs.85)?redirectedfrom=MSDN > (CaptureStackBackTrace) and its kernel/NTDLL equivalent > https://docs.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtlcapturestackbacktrace > (RtlCaptureStackBackTrace) > > > > 2) Alternatively, if more customization for the operation than what > CaptureStackBackTrace/RtlCaptureStackBackTrace affords is required, then > you could walk the stack manually by successively invoking > https://docs.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtlvirtualunwind > (RtlVirtualUnwind) for each frame, starting with a live register context > (e.g. as captured by > https://docs.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtlcapturecontext > (RtlCaptureContext)). This would give you the opportunity to examine, for > instance, the nonvolatile register context for each frame, albeit at the > cost of having to write platform dependent code for each build target (e.g. > AMD64, 32-bit ARMNT, ARM64, etc.). RtlCaptureStackBackTrace is essentially > implemented in this way for non-32-bit-x86 targets (x86 doesn't have what > most architectures would consider an "unwinder" in the sense of restoring > registers; there is simply a linked list of frames that participate in SEH > and that desire to be called for a dynamic unwind operation, so > RtlCaptureStackBackTrace assumes that EBP-based frames are in use and walks > an EBP-based frame chain on x86 - not all x86 code is written with > EBP-based frames so while even though we generally build the OS that way, > you might always run the risk of encountering external code that uses EBP > as a general purpose register for which such an unwind attempt for a stack > trace would fail). On non-x86 architectures, the various NT platform ABIs > typically strive to provide for precise unwinding at all frames and at > every instruction boundary where interrupts are enabled (at least assuming > that all code in the process is well-formed and follows the ABI properly > etc. - interrupts are always enabled in user mode code). For C/C++ code, > VC, ICC, and (I believe) LLVM/clang should support the platform > exception/unwind ABIs; GCC may or may not these days. > > > > The OS's exception and unwind dispatcher, for non-x86 build targets, uses > the virtual unwind facility to accomplish unwinding for purposes of > exception handling related exception dispatching & unwinding. > > > > > > On the non-x86 targets, debuggers and related tools would typically use > the platform ABI's exception and unwind handling structures for their > unwinding as well. Debuggers on x86 might either assume EBP-based frames, > or use compiler-specific debug information from debug symbols (typically > not embedded in the compiled binary for most NT-targeted code but rather > kept in a standalone .pdb/.dbg file etc.). > > > > > > Note that the platform exception and unwind APIs and associated structures > embedded in executable images don't, on their own, provide a way to map a > PC value to a meaningful symbolic function name, line number, etc.. If you > want to resolve PC values to function names, then you'll typically need > debug symbols for each module present on the call stack. For that, > provided that you have debug symbols available for all such frames (OS > binaries have these published on the Microsoft public symbol server; some > major third parties like web browsers publish their symbols on their own > symbol servers as well but many other third party binaries may not make > such affordances available). There are a set of APIs ( > https://docs.microsoft.com/en-us/windows/win32/debug/symbol-handling) to > load debug symbols for a module, and resolve a module + RVA (i.e. module + > offset into the module) to a symbol name and offset into the symbol. When > debug symbols aren't available, these facilities will typically pick the > nearest exported symbol (if any), which may or may not be at all meaningful > (most symbols aren't exported in most executables targeting NT). Note that > loading and processing debug symbols is typically a considerably more > "heavyweight" operation than invoking the platform unwind APIs. > > > > - S >Ian, how close are these capabilities to what you planned for libbacktrace? Than, please explain which scenarios did you assume, for libbacktrace, while integrating Ian's project with https://go.googlesource.com/gollvm/. Please check Ken's considerations and tell which of those would be the native way to support unwinding capabilities. I think there is some undocumented context, for that, related to gollvm. Consider this related to github issues, targeting support of Windows 10 (x86_64; i386). And btw - specifically ARM64 oriented Windows 10 haven't been suggested yet, by ARM's engineers, which contributed into brief support of ARM64 oriented Linux. Ivan On Sun, Aug 16, 2020 at 11:17 PM Ivan Serdyuk <local.tourist.kiev at gmail.com> wrote:> Martin, > good to hear from you. > Thanks for the effort - I will test on 32bit and 64bit Windows 10. > I will report ASAP. > > Ivan > > On Sun, Aug 16, 2020 at 8:42 PM Martin Storsjö <martin at martin.st> wrote: > >> On Sat, 15 Aug 2020, Ivan Serdyuk wrote: >> >> > >> > >> > On Sat, Aug 15, 2020 at 8:39 PM Martin Storsjö <martin at martin.st> >> wrote: >> > Hi, >> > >> > >> > On Sat, 15 Aug 2020, Ivan Serdyuk wrote: >> > >> > > Just as Shoaib said, libunwind only is useful in >> > environments >> > > that use >> > > the Itanium C++ ABI - there's really no use for it in an >> > MSVC >> > > context >> > > (either using MSVC or clang-cl to compile it). >> > > >> > > The particular linker error comes from the fact that >> > there's >> > > functions >> > > implemented in assembly, that expect the function name >> > to be >> > > mangled the >> > > itanium way, while the object files built by the >> > compiler expect >> > > the >> > > symbols to use the MSVC C++ name mangling, so there's an >> > > undefined >> > > reference. >> > > >> > > >> > > I see, thanks. >> > > >> > > Which options exist for MSVC, in sense an alternative to >> > libunwind and >> > > libbacktrace ? >> > >> > Well for libunwind, there's really no use for it in an MSVC >> > setting. All >> > the unwinding functionality is already built into the operating >> > system, >> > available via the Rtl*Unwind* functions. >> > >> > >> > Martin, >> > you mean these functions >> > >> https://docs.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtlunwind >> > >> > >> https://docs.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtlunwind >> > 2 >> > >> https://docs.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtlunwind >> > ex >> > >> https://docs.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtlvirtua >> > lunwind >> > ? >> >> Yes, those are the functions used for unwinding on windows. >> >> > For libbacktrace, I guess the _Unwind_Backtrace function in >> > libunwind >> > might work and be useful, even if the rest of libunwind doesn't >> > make sense >> > in such a context. >> >> Just FWIW, I made a RFC patch that fixes building libunwind in MSVC mode >> (still requiring clang-cl though), at https://reviews.llvm.org/D86041 - >> mostly just to show what it takes - not convinced that it necessarily is >> something that should be done. >> >> It does seem like _Unwind_Backtrace in libunwind doesn't work properly on >> x86_64 windows at the moment though, but on aarch64 it seems to do what >> one would expect. >> >> // Martin >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200816/6d4c6a45/attachment.html>
Than McIntosh via llvm-dev
2020-Aug-17 12:37 UTC
[llvm-dev] Supporting libunwind on Windows 10 (32bit; 64bit) for MSVC and Clang
>Than, >please explain which scenarios did you assume, for libbacktrace, whileintegrating Ian's project with https://go.googlesource.com/gollvm/. Not really sure I understand your question. By "scenarios" are you talking about strategies for doing stack unwinding? Gollvm targets libgo (the version of the Go runtime used by gccgo); the libgo runtime uses libbacktrace, but Gollvm is not doing anything special as relates to setting up or configuring libbacktrace. Thanks, Than On Sun, Aug 16, 2020 at 7:30 PM Ivan Serdyuk <local.tourist.kiev at gmail.com> wrote:> This is what Ken Johnson replied: > > On Sun, Aug 16, 2020 at 4:09 PM Skywing <Skywing at valhallalegends.com> > wrote: > >> I just briefly skimmed the discussion, so hopefully I didn't >> misunderstand what you are trying to do ... but if your objective is to >> capture a stack trace under NT, the OS exports several services for this >> purpose, depending on what you need out of such a stack trace : >> >> >> >> 1) There is a "pre-cooked" stack trace facility that will perform the >> entire stack trace for you (optionally skipping some frames at the top), >> returning an array of program counter values for each return address on the >> stack: >> https://docs.microsoft.com/en-us/previous-versions/windows/desktop/legacy/bb204633(v=vs.85)?redirectedfrom=MSDN >> (CaptureStackBackTrace) and its kernel/NTDLL equivalent >> https://docs.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtlcapturestackbacktrace >> (RtlCaptureStackBackTrace) >> >> >> >> 2) Alternatively, if more customization for the operation than what >> CaptureStackBackTrace/RtlCaptureStackBackTrace affords is required, then >> you could walk the stack manually by successively invoking >> https://docs.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtlvirtualunwind >> (RtlVirtualUnwind) for each frame, starting with a live register context >> (e.g. as captured by >> https://docs.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtlcapturecontext >> (RtlCaptureContext)). This would give you the opportunity to examine, for >> instance, the nonvolatile register context for each frame, albeit at the >> cost of having to write platform dependent code for each build target (e.g. >> AMD64, 32-bit ARMNT, ARM64, etc.). RtlCaptureStackBackTrace is essentially >> implemented in this way for non-32-bit-x86 targets (x86 doesn't have what >> most architectures would consider an "unwinder" in the sense of restoring >> registers; there is simply a linked list of frames that participate in SEH >> and that desire to be called for a dynamic unwind operation, so >> RtlCaptureStackBackTrace assumes that EBP-based frames are in use and walks >> an EBP-based frame chain on x86 - not all x86 code is written with >> EBP-based frames so while even though we generally build the OS that way, >> you might always run the risk of encountering external code that uses EBP >> as a general purpose register for which such an unwind attempt for a stack >> trace would fail). On non-x86 architectures, the various NT platform ABIs >> typically strive to provide for precise unwinding at all frames and at >> every instruction boundary where interrupts are enabled (at least assuming >> that all code in the process is well-formed and follows the ABI properly >> etc. - interrupts are always enabled in user mode code). For C/C++ code, >> VC, ICC, and (I believe) LLVM/clang should support the platform >> exception/unwind ABIs; GCC may or may not these days. >> >> >> >> The OS's exception and unwind dispatcher, for non-x86 build targets, uses >> the virtual unwind facility to accomplish unwinding for purposes of >> exception handling related exception dispatching & unwinding. >> >> >> >> >> >> On the non-x86 targets, debuggers and related tools would typically use >> the platform ABI's exception and unwind handling structures for their >> unwinding as well. Debuggers on x86 might either assume EBP-based frames, >> or use compiler-specific debug information from debug symbols (typically >> not embedded in the compiled binary for most NT-targeted code but rather >> kept in a standalone .pdb/.dbg file etc.). >> >> >> >> >> >> Note that the platform exception and unwind APIs and associated >> structures embedded in executable images don't, on their own, provide a way >> to map a PC value to a meaningful symbolic function name, line number, >> etc.. If you want to resolve PC values to function names, then you'll >> typically need debug symbols for each module present on the call stack. >> For that, provided that you have debug symbols available for all such >> frames (OS binaries have these published on the Microsoft public symbol >> server; some major third parties like web browsers publish their symbols on >> their own symbol servers as well but many other third party binaries may >> not make such affordances available). There are a set of APIs ( >> https://docs.microsoft.com/en-us/windows/win32/debug/symbol-handling) to >> load debug symbols for a module, and resolve a module + RVA (i.e. module + >> offset into the module) to a symbol name and offset into the symbol. When >> debug symbols aren't available, these facilities will typically pick the >> nearest exported symbol (if any), which may or may not be at all meaningful >> (most symbols aren't exported in most executables targeting NT). Note that >> loading and processing debug symbols is typically a considerably more >> "heavyweight" operation than invoking the platform unwind APIs. >> >> >> >> - S >> > > Ian, > how close are these capabilities to what you planned for libbacktrace? > > Than, > please explain which scenarios did you assume, for libbacktrace, while > integrating Ian's project with https://go.googlesource.com/gollvm/. > Please check Ken's considerations and tell which of those would be the > native way to support unwinding capabilities. > I think there is some undocumented context, for that, related to gollvm. > Consider this related to github issues, targeting support of Windows 10 > (x86_64; i386). And btw - specifically ARM64 oriented Windows 10 haven't > been suggested yet, by ARM's engineers, which contributed into brief > support of ARM64 oriented Linux. > > Ivan > > On Sun, Aug 16, 2020 at 11:17 PM Ivan Serdyuk < > local.tourist.kiev at gmail.com> wrote: > >> Martin, >> good to hear from you. >> Thanks for the effort - I will test on 32bit and 64bit Windows 10. >> I will report ASAP. >> >> Ivan >> >> On Sun, Aug 16, 2020 at 8:42 PM Martin Storsjö <martin at martin.st> wrote: >> >>> On Sat, 15 Aug 2020, Ivan Serdyuk wrote: >>> >>> > >>> > >>> > On Sat, Aug 15, 2020 at 8:39 PM Martin Storsjö <martin at martin.st> >>> wrote: >>> > Hi, >>> > >>> > >>> > On Sat, 15 Aug 2020, Ivan Serdyuk wrote: >>> > >>> > > Just as Shoaib said, libunwind only is useful in >>> > environments >>> > > that use >>> > > the Itanium C++ ABI - there's really no use for it in an >>> > MSVC >>> > > context >>> > > (either using MSVC or clang-cl to compile it). >>> > > >>> > > The particular linker error comes from the fact that >>> > there's >>> > > functions >>> > > implemented in assembly, that expect the function name >>> > to be >>> > > mangled the >>> > > itanium way, while the object files built by the >>> > compiler expect >>> > > the >>> > > symbols to use the MSVC C++ name mangling, so there's an >>> > > undefined >>> > > reference. >>> > > >>> > > >>> > > I see, thanks. >>> > > >>> > > Which options exist for MSVC, in sense an alternative to >>> > libunwind and >>> > > libbacktrace ? >>> > >>> > Well for libunwind, there's really no use for it in an MSVC >>> > setting. All >>> > the unwinding functionality is already built into the operating >>> > system, >>> > available via the Rtl*Unwind* functions. >>> > >>> > >>> > Martin, >>> > you mean these functions >>> > >>> https://docs.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtlunwind >>> > >>> > >>> https://docs.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtlunwind >>> > 2 >>> > >>> https://docs.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtlunwind >>> > ex >>> > >>> https://docs.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtlvirtua >>> > lunwind >>> > ? >>> >>> Yes, those are the functions used for unwinding on windows. >>> >>> > For libbacktrace, I guess the _Unwind_Backtrace function in >>> > libunwind >>> > might work and be useful, even if the rest of libunwind doesn't >>> > make sense >>> > in such a context. >>> >>> Just FWIW, I made a RFC patch that fixes building libunwind in MSVC mode >>> (still requiring clang-cl though), at https://reviews.llvm.org/D86041 - >>> mostly just to show what it takes - not convinced that it necessarily is >>> something that should be done. >>> >>> It does seem like _Unwind_Backtrace in libunwind doesn't work properly >>> on >>> x86_64 windows at the moment though, but on aarch64 it seems to do what >>> one would expect. >>> >>> // Martin >>> >>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200817/348e3a74/attachment.html>
Ian Lance Taylor via llvm-dev
2020-Aug-17 16:39 UTC
[llvm-dev] Supporting libunwind on Windows 10 (32bit; 64bit) for MSVC and Clang
Ivan Serdyuk <local.tourist.kiev at gmail.com> writes:> how close are these capabilities to what you planned for libbacktrace?To be clear, I don't personally plan to do anything. I agree that the capabilities mentioned in that e-mail are ones that libbacktrace would use on Windows when not building with GCC or clang. libbacktrace would also need more information from the debug info, such as mapping PC to file/line, which I didn't see mentioned. Ian