David Greene via llvm-dev
2021-Sep-23 15:42 UTC
[llvm-dev] [lit] XFAIL individual RUN lines?
We have a number of tests where we want to run the same code through
different subtargets:
# RUN: llc -mcpu=<subtarget1> ...
# RUN: llc -mcpu=<subtarget2> ...
Not all subtargets are expected to pass, but some will pass. We'd like
to know which tests are passing for which subtargets. We don't want to
XFAIL the whole test (because it would stop testing of working
subtargets) but we do what to know which subtargets need work.
Is it possible to somehow XFAIL an individual RUN line? The
documentation seems to imply it's all-or-nothing.
Alternatively, is it possible to get a test summary that shows PASS/FAIL
for individual RUN lines rather than for the test as a whole?
If neither of these is possible, would patches to implement them be
welcome?
Thanks for your help!
-David
David Spickett via llvm-dev
2021-Sep-23 15:51 UTC
[llvm-dev] [lit] XFAIL individual RUN lines?
Sounds a lot like https://lists.llvm.org/pipermail/llvm-dev/2021-September/152829.html With the addition of being able to tell lit to continue if a line does fail. I'd find both features quite useful. On Thu, 23 Sept 2021 at 16:42, David Greene via llvm-dev <llvm-dev at lists.llvm.org> wrote:> > We have a number of tests where we want to run the same code through > different subtargets: > > # RUN: llc -mcpu=<subtarget1> ... > # RUN: llc -mcpu=<subtarget2> ... > > Not all subtargets are expected to pass, but some will pass. We'd like > to know which tests are passing for which subtargets. We don't want to > XFAIL the whole test (because it would stop testing of working > subtargets) but we do what to know which subtargets need work. > > Is it possible to somehow XFAIL an individual RUN line? The > documentation seems to imply it's all-or-nothing. > > Alternatively, is it possible to get a test summary that shows PASS/FAIL > for individual RUN lines rather than for the test as a whole? > > If neither of these is possible, would patches to implement them be > welcome? > > Thanks for your help! > > -David > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Stephen Neuendorffer via llvm-dev
2021-Sep-23 18:21 UTC
[llvm-dev] [lit] XFAIL individual RUN lines?
I've wanted something similar, but for REQUIRES: lines, so that each RUN line could have a different set of required features in order to execute. Steve On Thu, Sep 23, 2021 at 8:42 AM David Greene via llvm-dev < llvm-dev at lists.llvm.org> wrote:> We have a number of tests where we want to run the same code through > different subtargets: > > # RUN: llc -mcpu=<subtarget1> ... > # RUN: llc -mcpu=<subtarget2> ... > > Not all subtargets are expected to pass, but some will pass. We'd like > to know which tests are passing for which subtargets. We don't want to > XFAIL the whole test (because it would stop testing of working > subtargets) but we do what to know which subtargets need work. > > Is it possible to somehow XFAIL an individual RUN line? The > documentation seems to imply it's all-or-nothing. > > Alternatively, is it possible to get a test summary that shows PASS/FAIL > for individual RUN lines rather than for the test as a whole? > > If neither of these is possible, would patches to implement them be > welcome? > > Thanks for your help! > > -David > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210923/c69a6210/attachment.html>