I was thinking of something along the lines of:
// SCRIPT-POSIX: posix/aggregate-indirect-arg.s
// SCRIPT-WIN: win/aggregate-indirect-arg.s
class SVal {
public:
~SVal() {}
const void* Data;
unsigned Kind;
};
void bar(SVal &v) {}
class A {
public:
void foo(SVal v) { bar(v); }
};
int main() {
SVal v;
v.Data = 0;
v.Kind = 2142;
A a;
a.foo(v);
return 0;
}
Then, you could have:
// posix/aggregate-indirect-arg.s
// RUN: %clangxx %target_itanium_abi_host_triple -O0 -g %i -c -o %t.o
// RUN: %clangxx %target_itanium_abi_host_triple %t.o -o %t.out
// RUN: %test_debuginfo %s %t.out
// Radar 8945514
// DEBUGGER: break 22
// DEBUGGER: r
// DEBUGGER: p v
// CHECK: ${{[0-9]+}} // CHECK: Data ={{.*}} 0x0{{(0*)}}
// CHECK: Kind = 2142
// win/aggregate-indirect-arg.s
// RUN: %clangcl /Z7 %i /c /Fo%t.obj
// RUN: %lld-link /DEBUG %t.obj /out:%t.lld.exe
// RUN: %run_windbg %t.lld.exe %s
// RUN: %ms-link /DEBUG:FASTLINK %t.obj /out:%t.fastlink.exe
// RUN: %run_windbg %t.fastlink.exe %s
// DEBUGGER: bp 22
// DEBUGGER: g
// DEBUGGER: dt v
// CHECK: Local var {{.*}} Type SVal
// CHECK: +0x000 Data : (null)
// CHECK: +0x004 Kind : 0x85e
Eventually, some tests will inevitably need to Windows or Posix specific,
so you're going to have to have all this extra stuff (the new
substitutions, the different command lines, the custom output formats,
etc. So I think something like this provides maximal encouragement of
sharing whenever possible (since you can almost always share source code),
while still allowing each format to test real input and real output.
In rare cases where command lines, input, and output formats match up, just
replace the SCRIPT-POSIX and SCRIPT-WIN directives with the actual run and
check lines.
On Thu, Sep 7, 2017 at 12:41 PM Adrian Prantl <aprantl at apple.com>
wrote:
> On Sep 7, 2017, at 12:31 PM, Zachary Turner <zturner at google.com>
wrote:
>
> That's true, but it still requires the person writing the test to be
> familiar with both formats. Also anything more trivial than dumping a
> single local variable will probably run into even more issues.
>
> For example, The expression parsers each have their own unique quirks, and
> it might not always be easy to translate.
>
>
> The vocabulary used by the tests is extremely limited, it is basically
"b
> <line>", "r", "p <var>",
"c", and twice "ptype" (which I would guess may be
> hard to support).
> Do any of these lack an equivalent under WinDbg?
>
>
> Besides all the format compatibility issues, there's the more general
> issue that massaging one format into another format just introduces another
> place for things to go wrong and/or be flaky. But as we are the ones that
> have to do the massaging, we get the short end of the stick.
>
> One possible bridge is to make some sort of special directive that
> identifies a certain fragment of the test file as gcc input & checks,
and
> another directive that identifies windbg input & checks.
>
>
> One way to implement this (especially if it only affects few tests) is to
> mark the test as unsupported under windows and then have a test script in,
> e.g., the pdb directory that has a new RUN command and CHECK lines but uses
> the same source code.
>
>
> This way each could have their own native run lines, debugger commands,
> and check statements, but share the same source code. The runner then
> chooses which mode to run based on which directives are present and whether
> a debugger to run the corresponding fragment in is found.
>
>
> -- adrian
>
>
> This way you could have one or both directives in a file, and we'd run
> whichever version(s) are appropriate
> On Thu, Sep 7, 2017 at 12:19 PM Adrian Prantl <aprantl at apple.com>
wrote:
>
>> On Sep 7, 2017, at 12:11 PM, Adrian Prantl <aprantl at apple.com>
wrote:
>>
>>
>> On Sep 7, 2017, at 12:01 PM, Zachary Turner <zturner at
google.com> wrote:
>>
>>
>>
>> On Thu, Sep 7, 2017 at 11:49 AM Adrian Prantl <aprantl at
apple.com> wrote:
>>
>>> On Sep 7, 2017, at 11:37 AM, Zachary Turner via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>> To be clear, the tests I'm proposing will have no resemblance
whatsoever
>>> to GDB, so I would be intentionally forking the set of tests in
this
>>> regards. So there would very clearly be a paradigm shift in
writing
>>> CodeView debug info tests (which would be written in JavaScript
using
>>> WinDbg specific debugger commands) and DWARF debug info tests
(which would
>>> be written in whatever / using GDB style commands)
>>>
>>>
>>> That sounds like an unfortunate direction. Can you explain why it
>>> wouldn't be possible to write a wrapper (in JavaScript) that
interprets the
>>> 3ish gdb commands used by the tests in terms of WinDbg? Similar to
how LLDB
>>> is supported?
>>>
>>>
>> I can think of a couple of reasons:
>>
>> 1) We're already going to need entirely different runlines. clang
and
>> clang-cl don't use the same command line options, or for that
matter even
>> styles.
>>
>>
>> I'm not familiar with clang-cl so please bear with me if this is a
stupid
>> question: Is clang-cl just an MSVC compatible driver? Can you not
produce
>> the same result by using the "standard" clang driver and a
windows target
>> triple?
>>
>>
>> 2) The output format is going to be different. whereas the current
tests
>> look for things like
>>
>>
>> If you are going to write a wrapper script for the debugger commands,
>> that wrapper script could also transform the output to look more like
the
>> GDB output that the CHECK-lines are looking for.
>>
>>
>> // CHECK: ${{[0-9]+}} >> // CHECK: Data ={{.*}} 0x0{{(0*)}}
>> // CHECK: Kind = 2142
>>
>> In WinDbg this is going to be more like:
>>
>> Local var @ 0x6ffa50 Type SVal
>>
>> +0x000 Data : (null) +0x004 Kind : 0x85e
>>
>>
>>
>> Alternatively, this is still similar enough that we could relax the
check
>> to look like
>> CHECK: Data{{.*[:=].*(0x0|.null.)}}
>> CHECK: Kind{{.*[:=].*(2142|0x85e)}}
>> without much loss.
>>
>> -- adrian
>>
>> So we're also going to need different check lines.
>>
>> At this point, the only similarities in the PDB / DWARF tests is going
to be the source code, as I don't see an easy way to automatically translate
command lines, input commands, and output text.
>>
>> By the way, I'm fine with adding, e.g., a PDB subdirectory with a
lit
>> filter that contains tests that will only work under Windows, but it
would
>> be very unfortunate if there was no common set of tests, since then the
>> Windows folks wouldn't benefit from new tests being added for GDB
and vice
>> versa.
>>
>> -- adrian
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170907/0f65b966/attachment.html>
> On Sep 7, 2017, at 1:15 PM, Zachary Turner <zturner at google.com> wrote: > > I was thinking of something along the lines of: > > // SCRIPT-POSIX: posix/aggregate-indirect-arg.s > // SCRIPT-WIN: win/aggregate-indirect-arg.s > > class SVal { > public: > ~SVal() {} > const void* Data; > unsigned Kind; > }; > > void bar(SVal &v) {} > class A { > public: > void foo(SVal v) { bar(v); } > }; > > int main() { > SVal v; > v.Data = 0; > v.Kind = 2142; > A a; > a.foo(v); > return 0; > } > > Then, you could have: > > // posix/aggregate-indirect-arg.s > > // RUN: %clangxx %target_itanium_abi_host_triple -O0 -g %i -c -o %t.o > // RUN: %clangxx %target_itanium_abi_host_triple %t.o -o %t.out > // RUN: %test_debuginfo %s %t.out > // Radar 8945514 > // DEBUGGER: break 22 > // DEBUGGER: r > // DEBUGGER: p v > // CHECK: ${{[0-9]+}} > // CHECK: Data ={{.*}} 0x0{{(0*)}} > // CHECK: Kind = 2142 > > > // win/aggregate-indirect-arg.s > > // RUN: %clangcl /Z7 %i /c /Fo%t.obj > // RUN: %lld-link /DEBUG %t.obj /out:%t.lld.exe > // RUN: %run_windbg %t.lld.exe %s > // RUN: %ms-link /DEBUG:FASTLINK %t.obj /out:%t.fastlink.exe > // RUN: %run_windbg %t.fastlink.exe %s > // DEBUGGER: bp 22 > // DEBUGGER: g > // DEBUGGER: dt v > // CHECK: Local var {{.*}} Type SVal > // CHECK: +0x000 Data : (null) > // CHECK: +0x004 Kind : 0x85e > > Eventually, some tests will inevitably need to Windows or Posix specific, so you're going to have to have all this extra stuff (the new substitutions, the different command lines, the custom output formats, etc. So I think something like this provides maximal encouragement of sharing whenever possible (since you can almost always share source code), while still allowing each format to test real input and real output.I understand the desire to allow for Windows-specific tests and I think it would be good to add them to the repository in a windows subdirectory. Looking at the example you posted, the two variants are so structurally similar that I believe it would be a better to come up with a common abstraction from a readability / maintenance effort perspective. Basically, the only thing that the RUN lines do is compile and link executables from source code using the default target and run the test_debuginfo command. I think it would be better to define a new command substitution %clang-compile-link(?) in LIT that has different implementations on windows and posix. The set of debugger commands used by the tests is so tiny that it should not be a lot of work to implement a wrapper for the windows debugger (it took me about a day to write the python wrapper for LLDB including learning how to use the Python API) and it should also be possible to either do a sed-style massaging of the output or relax the CHECKs to work with both formats. I really want to avoid duplicating the debugger commands and checks, and I also want to maintain the ability to put the commands and CHECKs into the source code, since this makes the tests much easier to understand. Using a common abstraction will save us a lot of time in the long run, make maintenance and adding new tests cheaper, and won't prevent you from also having windows-specific tests that may use an expanded vocabulary. What do you think? -- adrian
On Fri, Sep 8, 2017 at 9:00 AM Adrian Prantl <aprantl at apple.com> wrote:> > > Eventually, some tests will inevitably need to Windows or Posix > specific, so you're going to have to have all this extra stuff (the new > substitutions, the different command lines, the custom output formats, > etc. So I think something like this provides maximal encouragement of > sharing whenever possible (since you can almost always share source code), > while still allowing each format to test real input and real output. > > I understand the desire to allow for Windows-specific tests and I think it > would be good to add them to the repository in a windows subdirectory. > > Looking at the example you posted, the two variants are so structurally > similar that I believe it would be a better to come up with a common > abstraction from a readability / maintenance effort perspective. Basically, > the only thing that the RUN lines do is compile and link executables from > source code using the default target and run the test_debuginfo command. I > think it would be better to define a new command substitution > %clang-compile-link(?) in LIT that has different implementations on windows > and posix. The set of debugger commands used by the tests is so tiny that > it should not be a lot of work to implement a wrapper for the windows > debugger (it took me about a day to write the python wrapper for LLDB > including learning how to use the Python API) and it should also be > possible to either do a sed-style massaging of the output or relax the > CHECKs to work with both formats. > > I really want to avoid duplicating the debugger commands and checks, and I > also want to maintain the ability to put the commands and CHECKs into the > source code, since this makes the tests much easier to understand. Using a > common abstraction will save us a lot of time in the long run, make > maintenance and adding new tests cheaper, and won't prevent you from also > having windows-specific tests that may use an expanded vocabulary. > > What do you think? > -- adrian >I understand the desire to keep them as similar as possible, but I'm still not really sold that massaging fickle text output into a different text format is going to make things more scalable. I'd like there to be as few layers of text processing as possible. If someone files a bug report that includes a WinDbg command log, I'd like to be able to paste those statements into a test. I also expect that on Windows we will end up having far more debug info tests than other platforms, specifically because we don't have the ability to write tests against the debugger (as it's proprietary / closed source). So the language used in the current set of debug info tests is very simple, because GDB and LLDB already have test suites that test more complicated things. But the problem is, we don't have those other test suites to fall back on, so we will need much more. For example, we may end up wanting a test that exercises custom debug visualizers, or a test that a certain proprietary debugger feature works, or a test that builtin debug visualizers of STL types work. To write a check for the latter, you need to know the layout of the type, which depends on the standard library implementation, so it's already going to be different. If all we're doing is printing an integer, then I agree we can write a common test. But I don't think this is going to be the case outside of 1 or 2 trivial tests. There's also the issue that we may want to test entirely different things. For example, in the hypothetical example I posted earlier, we compile once and link twice with 2 different linkers. But we might even want to compile twice and link twice (compile same program with cl and clang-cl, then link both with lld). The fundamental difference here is that we have two different things that can emit debug info - the compiler and linker - and we need the flexibility to test both independently of each other. On a posixy platform, you only care the compiler and don't care what the linker is. On the other hand, it has its own set of unique aspects. You might decide to compile and link many times, so that you can test -gsplit-dwarf, -gdebug-info-kind=limited, -gdebug-info-kind=full, -gdb-index, etc. against a single program. I don't see a useful abstraction that glosses over these differences that isn't a ton of work for minimal gain, given the frequency with which we'd need to fall back to a custom test anyway. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170908/2a9c5966/attachment.html>