What is the status of debuginfo-tests? Is it actively supported? How do you run it? It doesn't appear to be based on lit, any particular reason? Why is it its own repo instead of being part of llvm repo? I'd like improve this to support CodeView and PDB, such that it would only run on Windows and only if a suitable debugger was found (probably WinDbg). WinDbg supports a JavaScript-based scripting model, similar to how LLDB supports a Python based model, so my thoughts were to have a lit-based runner that scans for .js files that contain a test script alongside some source, then build the program, run it in WinDbg with some script that does various things, and exits the debugger, moving on to the next test. Anything I should be aware of / careful of when messing around in here? And any reason it can't be moved to llvm/tests and ported to lit? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170907/0ee2d73f/attachment.html>
It's used, but not a huge repository of things, as you can see. I think I've run it once or twice, but a long time ago. It was introduced for/by Apple/LLDB stuff, so it's not something I've paid much attention to. It's probably not suitable as part of llvm tests directly. Those tests are designed to be shorter/narrower/more focussed than full integration tests (we don't execute any compiled programs under test there, for example). Porting to lit seems probably fine/good. On Thu, Sep 7, 2017 at 11:23 AM Zachary Turner <zturner at google.com> wrote:> What is the status of debuginfo-tests? Is it actively supported? How do > you run it? It doesn't appear to be based on lit, any particular reason? > Why is it its own repo instead of being part of llvm repo? > > I'd like improve this to support CodeView and PDB, such that it would only > run on Windows and only if a suitable debugger was found (probably > WinDbg). WinDbg supports a JavaScript-based scripting model, similar to > how LLDB supports a Python based model, so my thoughts were to have a > lit-based runner that scans for .js files that contain a test script > alongside some source, then build the program, run it in WinDbg with some > script that does various things, and exits the debugger, moving on to the > next test. > > Anything I should be aware of / careful of when messing around in here? > And any reason it can't be moved to llvm/tests and ported to lit? >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170907/e5d0835a/attachment.html>
Even if it were under llvm, that doesn't mean it would have to be part of ninja check-all. Currently we have lldb/test and lldb/unittests, what about a third lldb/dbgtests? Then you could run it as ninja check-dbgtest. Although now that I think about it, we would want it to be able to invoke clang-cl directly, so I guess that does necessitate it *not* being under llvm. On Thu, Sep 7, 2017 at 11:26 AM David Blaikie <dblaikie at gmail.com> wrote:> It's used, but not a huge repository of things, as you can see. I think > I've run it once or twice, but a long time ago. It was introduced for/by > Apple/LLDB stuff, so it's not something I've paid much attention to. > > It's probably not suitable as part of llvm tests directly. Those tests are > designed to be shorter/narrower/more focussed than full integration tests > (we don't execute any compiled programs under test there, for example). > > Porting to lit seems probably fine/good. > > On Thu, Sep 7, 2017 at 11:23 AM Zachary Turner <zturner at google.com> wrote: > >> What is the status of debuginfo-tests? Is it actively supported? How do >> you run it? It doesn't appear to be based on lit, any particular reason? >> Why is it its own repo instead of being part of llvm repo? >> >> I'd like improve this to support CodeView and PDB, such that it would >> only run on Windows and only if a suitable debugger was found (probably >> WinDbg). WinDbg supports a JavaScript-based scripting model, similar to >> how LLDB supports a Python based model, so my thoughts were to have a >> lit-based runner that scans for .js files that contain a test script >> alongside some source, then build the program, run it in WinDbg with some >> script that does various things, and exits the debugger, moving on to the >> next test. >> >> Anything I should be aware of / careful of when messing around in here? >> And any reason it can't be moved to llvm/tests and ported to lit? >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170907/2e3b5413/attachment.html>
As executable tests, they fail spectacularly in a cross-build environment. And you need some sort of debugger with GDB-like commands and output (or have some front end to your debugger that imitates that) in order to run them. I think they would need to stay in a separate project because of those requirements. --paulr From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of David Blaikie via llvm-dev Sent: Thursday, September 07, 2017 11:26 AM To: Zachary Turner; llvm-dev Subject: Re: [llvm-dev] Status of debuginfo-tests It's used, but not a huge repository of things, as you can see. I think I've run it once or twice, but a long time ago. It was introduced for/by Apple/LLDB stuff, so it's not something I've paid much attention to. It's probably not suitable as part of llvm tests directly. Those tests are designed to be shorter/narrower/more focussed than full integration tests (we don't execute any compiled programs under test there, for example). Porting to lit seems probably fine/good. On Thu, Sep 7, 2017 at 11:23 AM Zachary Turner <zturner at google.com<mailto:zturner at google.com>> wrote: What is the status of debuginfo-tests? Is it actively supported? How do you run it? It doesn't appear to be based on lit, any particular reason? Why is it its own repo instead of being part of llvm repo? I'd like improve this to support CodeView and PDB, such that it would only run on Windows and only if a suitable debugger was found (probably WinDbg). WinDbg supports a JavaScript-based scripting model, similar to how LLDB supports a Python based model, so my thoughts were to have a lit-based runner that scans for .js files that contain a test script alongside some source, then build the program, run it in WinDbg with some script that does various things, and exits the debugger, moving on to the next test. Anything I should be aware of / careful of when messing around in here? And any reason it can't be moved to llvm/tests and ported to lit? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170907/2dc6d2d6/attachment.html>
> On Sep 7, 2017, at 11:23 AM, Zachary Turner <zturner at google.com> wrote: > > What is the status of debuginfo-tests? Is it actively supported?Yes absolutely. For example, all jobs on green dragon run it.> How do you run it?The repository is supposed to be cloned into llvm/tools/clang/test and will then appear as part of check-clang.> It doesn't appear to be based on lit, any particular reason?It is most definitely based on lit.> Why is it its own repo instead of being part of llvm repo?Presumably (this decision predates me) so it can be easily disabled.> I'd like improve this to support CodeView and PDB, such that it would only run on Windows and only if a suitable debugger was found (probably WinDbg). WinDbg supports a JavaScript-based scripting model, similar to how LLDB supports a Python based model, so my thoughts were to have a lit-based runner that scans for .js files that contain a test script alongside some source, then build the program, run it in WinDbg with some script that does various things, and exits the debugger, moving on to the next test. >The repository already supports running with gdb and lldb (by implementing the gdb commands used in the tests using the lldb python API), so I'm sure a similar approach should work for Windows as well.> Anything I should be aware of / careful of when messing around in here? And any reason it can't be moved to llvm/tests and ported to lit?As I said above, it is based on lit and considered part of the clang testsuite. -- adrian
> On Sep 7, 2017, at 11:26 AM, David Blaikie <dblaikie at gmail.com> wrote: > > It's used, but not a huge repository of things, as you can see. I think I've run it once or twice, but a long time ago. It was introduced for/by Apple/LLDB stuff, so it's not something I've paid much attention to.To clarify, they have been introduces by Apple as an end-to-end test for clang's debug info under GDB and I later added a wrapper script so they also work with LLDB.> > It's probably not suitable as part of llvm tests directly. Those tests are designed to be shorter/narrower/more focussed than full integration tests (we don't execute any compiled programs under test there, for example). > > Porting to lit seems probably fine/good.As mentioned in another reply, they already use lit.> On Thu, Sep 7, 2017 at 11:23 AM Zachary Turner <zturner at google.com <mailto:zturner at google.com>> wrote: > What is the status of debuginfo-tests? Is it actively supported? How do you run it? It doesn't appear to be based on lit, any particular reason? Why is it its own repo instead of being part of llvm repo? > > I'd like improve this to support CodeView and PDB, such that it would only run on Windows and only if a suitable debugger was found (probably WinDbg). WinDbg supports a JavaScript-based scripting model, similar to how LLDB supports a Python based model, so my thoughts were to have a lit-based runner that scans for .js files that contain a test script alongside some source, then build the program, run it in WinDbg with some script that does various things, and exits the debugger, moving on to the next test. > > Anything I should be aware of / careful of when messing around in here? And any reason it can't be moved to llvm/tests and ported to lit?-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170907/9a296107/attachment.html>
On Thu, Sep 7, 2017 at 11:41 AM Adrian Prantl <aprantl at apple.com> wrote:> > > On Sep 7, 2017, at 11:23 AM, Zachary Turner <zturner at google.com> wrote: > > How do you run it? > > The repository is supposed to be cloned into llvm/tools/clang/test and > will then appear as part of check-clang. > > > It doesn't appear to be based on lit, any particular reason? > > It is most definitely based on lit. >Hmm, I'm using a mono-repo and it appears under a top-level debuginfo-tests folder that doesn't have any lit configuration files. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170907/40b43a36/attachment.html>