search for: update_llc_test_checks

I've run update_llc_test_checks on a set of tests and am seeing some weird behavior. The CHECK lines appear directly after the function's first line, even if there are multiple arguments. E.g.: define <vscale x 4 x i32> @sel_nxv4i32(<vscale x 4 x i1> %p, ; CHECK-LABEL: sel_nxv4i32: ; CHECK: // %bb.0: ; CHEC...

There are multiple problems/questions here: 1. Make sure you've updated trunk to the latest rev before running update_llc_test_checks.py on lea-3.ll. Ie, I would only expect the output you're seeing if you're running the script on a version of that test file before r313631. After that commit, each RUN has its own check prefix, so there should be no conflict opportunity. 2. I didn't realize the scope of the patch cove...

...t; obvious for x86 asm, so feel free to just post those diffs in an updated > version of the patch at D37896. > > If the test files have auto-generated assertions (look for this string > on the first line of the test file: "NOTE: Assertions have been > autogenerated by utils/update_llc_test_checks.py"... > and both of these do as of: https://reviews.llvm.org/rL313631 > <https://reviews.llvm.org/rL313631> ), then it's easy to observe the > diffs by re-running that script after your code patch is applied: > $ /path/to/update_llc_test_checks.py --llc=/path/to/local/...

...on and the unreachable is trigged. Is this really working as intended or am I missing something here? Cheers, Michael On 20.09.2017 16:10, Sanjay Patel wrote: > There are multiple problems/questions here: > > 1. Make sure you've updated trunk to the latest rev before running > update_llc_test_checks.py on lea-3.ll. Ie, I would only expect the > output you're seeing if you're running the script on a version of that > test file before r313631. After that commit, each RUN has its own check > prefix, so there should be no conflict opportunity. > > 2. I didn't realize...

...is is unfortunately not always obvious for x86 asm, so feel free to just post those diffs in an updated version of the patch at D37896. If the test files have auto-generated assertions (look for this string on the first line of the test file: "NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py"... and both of these do as of: https://reviews.llvm.org/rL313631 ), then it's easy to observe the diffs by re-running that script after your code patch is applied: $ /path/to/update_llc_test_checks.py --llc=/path/to/local/and/new/llc lea-3.ll $ svn diff lea-3.ll On Tue, Sep 19, 2017 a...

...his really working as intended or am I missing something here? > > Cheers, > Michael > > On 20.09.2017 16:10, Sanjay Patel wrote: > > There are multiple problems/questions here: > > > > 1. Make sure you've updated trunk to the latest rev before running > > update_llc_test_checks.py on lea-3.ll. Ie, I would only expect the > > output you're seeing if you're running the script on a version of that > > test file before r313631. After that commit, each RUN has its own check > > prefix, so there should be no conflict opportunity. > > > > 2....

...CHECK-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0 > ---------------- > greened wrote: >> Can we make this test less brittle by using FileCheck variables? >> This goes for pretty much every test in this patch. > I'm sorry but no - its been repeatedly proven that using > update_llc_test_checks.py on the majority of x86 tests is the way > forward - it speeds up creation of tests (x86 by far has the highest > test coverage), makes regeneration of checks trivial and it prevents > dodgy code being 'hidden' (either on purpose or by > accident). Additionally many x86 subtar...

...am I missing something here? > > Cheers, > Michael > > On 20.09.2017 16:10, Sanjay Patel wrote: > > There are multiple problems/questions here: > > > > 1. Make sure you've updated trunk to the latest rev before running > > update_llc_test_checks.py on lea-3.ll. Ie, I would only expect the > > output you're seeing if you're running the script on a version of > that > > test file before r313631. After that commit, each RUN has its own > check > > prefix, so there should be no conflict op...

...is is unfortunately not always obvious for x86 asm, so feel free to just post those diffs in an updated version of the patch at D37896. If the test files have auto-generated assertions (look for this string on the first line of the test file: "NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py"... and both of these do as of: https://reviews.llvm.org/rL313631 ), then it's easy to observe the diffs by re-running that script after your code patch is applied: $ /path/to/update_llc_test_checks.py --llc=/path/to/local/and/new/llc lea-3.ll $ svn diff lea-3.ll On Tue, Sep 19, 2017 a...

...be problematic for very large test cases that are likely to see churn due to scheduling or regallloc changes. Being able to regenerate the CHECK lines and view the diff is also incredibly helpful when rebasing or moving a patch between different branches. My policy for test/Codegen/RISCV is to use update_llc_test_checks.py wherever possible, except in cases where there are so many CHECK lines on the output that they obscure the property being tested, indicating that a more limited hand-crafted pattern would be superior. Best, Alex

...large test cases that are likely to see churn due to scheduling or > regallloc changes. Being able to regenerate the CHECK lines and view > the diff is also incredibly helpful when rebasing or moving a patch > between different branches. > > My policy for test/Codegen/RISCV is to use update_llc_test_checks.py > wherever possible, except in cases where there are so many CHECK lines > on the output that they obscure the property being tested, indicating > that a more limited hand-crafted pattern would be superior. > > Best, > > Alex > ____________________________________________...

Hello llvm-dev, On a recent code review I was asked to auto-generate assertion checks for my unit test. I wasn't aware that this was even possible. I am referring to the python `update` scripts under `utils` directory. My first reaction was wow! I found it very practical and useful. It saves you significant amount of time when writing a regression test. So I gave it a try. The generated

...for j in s] for i, s in enumerate(shuffles)] if len(results) != 1: - print >>sys.stderr, 'ERROR: Bad results: %s' % (results,) + print('ERROR: Bad results: %s' % (results,), file=sys.stderr) sys.exit(1) result = results[0] Index: update_llc_test_checks.py =================================================================== --- update_llc_test_checks.py (revision 249819) +++ update_llc_test_checks.py (working copy) @@ -6,7 +6,7 @@ FileCheck patterns. It can either update all of the tests in the file or a single test function. """...

...ocks in the same test > differently each time? > If it is *NOT* deterministic, then it is most certainly a bug that > should be reported. > If it is deterministic, (answer without backend-specific knowledge) it > might be best not to worry about this, > and just use the llvm/utils/update_llc_test_checks.py to simplify > updating of the tests. > >> Can anyone offer advice? >> >> Carl > Roman. > >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> https://lists.llvm.org/cgi-bin/mai...

...jority are not. A test that, for example, checks the entire generated asm for a match is indeed very fragile. A test that checks whether a specific instruction/mnemonic was emitted is generally not, at least in my experience. End-to-end tests require some care in construction. I don't think update_llc_test_checks.py-type operation is desirable. Still, you raise a valid point and I think present some good options below. > A couple of approaches we could consider: > > 1. Simply restrict end to end tests to crash/assert cases.  (i.e. no > property of the generated code is checked, other than...

...t are likely to see churn due to scheduling or >> regallloc changes. Being able to regenerate the CHECK lines and view >> the diff is also incredibly helpful when rebasing or moving a patch >> between different branches. >> >> My policy for test/Codegen/RISCV is to use update_llc_test_checks.py >> wherever possible, except in cases where there are so many CHECK lines >> on the output that they obscure the property being tested, indicating >> that a more limited hand-crafted pattern would be superior. >> >> Best, >> >> Alex >> ____________...

...churn due to scheduling or >>> regallloc changes. Being able to regenerate the CHECK lines and view >>> the diff is also incredibly helpful when rebasing or moving a patch >>> between different branches. >>> >>> My policy for test/Codegen/RISCV is to use update_llc_test_checks.py >>> wherever possible, except in cases where there are so many CHECK lines >>> on the output that they obscure the property being tested, indicating >>> that a more limited hand-crafted pattern would be superior. >>> >>> Best, >>> >>&gt...

I am currently improving the D37896 to include the suggestions from Chad. However, running the lit checks for the x86 backend I observe some changes in the generated MC, e.g.: llvm/test/CodeGen/X86/lea-3.ll:13:10: error: expected string not found in input ; CHECK: leal ([[A0]],[[A0]],2), %eax ^ <stdin>:10:2: note: scanning from here orq %rdi, %rax ^ <stdin>:10:2:

...ing or >>>> regallloc changes. Being able to regenerate the CHECK lines and view >>>> the diff is also incredibly helpful when rebasing or moving a patch >>>> between different branches. >>>> >>>> My policy for test/Codegen/RISCV is to use update_llc_test_checks.py >>>> wherever possible, except in cases where there are so many CHECK lines >>>> on the output that they obscure the property being tested, indicating >>>> that a more limited hand-crafted pattern would be superior. >>>> >>>> Best, >...

...vm.org/rL290346). * GlobalISel implementation work continues, with the addition of a basic tablegen backend. [r290284](http://reviews.llvm.org/rL290284). * A POWER9 processor model was added for scheduling. [r290102](http://reviews.llvm.org/rL290102). * ARM and PPC support has been added to the `update_llc_test_checks.py` script. [r290144](http://reviews.llvm.org/rL290144), [r290370](http://reviews.llvm.org/rL290370). * The GlobPattern class has been imported into LLVM from LLD. [r290212](http://reviews.llvm.org/rL290212). * Codegen for the x86 vectorcall calling convention has landed. [r290240](http://reviews...