Hi David,
Unfortunately writing a reliable test is tricky given that the
functionality we're trying to test involves timing. I would advise
against disabling the test entirely because it actually tests
functionality that people use. I'd suggest bumping up the time limits.
This is what I've done in the past. See
commit 6dfcc78364fa3e8104d6e6634733863eb0bf4be8
Author: Dan Liew <dan at su-root.co.uk>
Date: Tue May 22 15:06:29 2018 +0000
[lit] Try to make `shtest-timeout.py` test more reliable by using a
larger timeout value. This really isn't very good because it will
still be susceptible to machine performance.
While we are here also fix a bug in validation of
`maxIndividualTestTime` where previously it wasn't checked if the
type was an int.
rdar://problem/40221572
llvm-svn: 332987
HTH,
Dan.
On Wed, 16 Sep 2020 at 09:37, David Blaikie <dblaikie at gmail.com>
wrote:>
> Ping on this
>
> On Wed, Sep 9, 2020 at 8:27 PM David Blaikie <dblaikie at gmail.com>
wrote:
> >
> > The clang-cmake-armv8-lld (linaro-toolchain owners) buildbot is timing
out trying to run some timeout tests (Dan Liew author):
> >
> > Pass:
http://lab.llvm.org:8011/builders/clang-cmake-armv8-lld/builds/5672
> > Fail:
http://lab.llvm.org:8011/builders/clang-cmake-armv8-lld/builds/5673
> >
> > Is there anything we can do to the buildbot? Or the tests? (bump up
the time limits or maybe remove the tests as unreliable?)