llvm dev - Jul 2013 - [LLVMdev] [RFC] Switching make check to use 'set -o pipefail'

On 5 July 2013 00:15, Stephen Lin <swlin at post.harvard.edu>
wrote:
> Hi Rafael,
>
> I think you saw my other e-mail, but just in case you haven't, do you
> have any thoughts about making this an option that could be easily
> disabled on the command line without maintaing a patch to lit? I think
> it would help out-of-tree target maintainers to transition, since I'm
> sure there will be a lot of similarly broken tests to fix.
I don't think it is all that many since it was less than one day of
work for the in tree ones. But if there is the desire for such an
option I can try to add it. What should I use? An environment
variable?
> Stephen
>
Cheers,
Rafael

> I don't think it is all that many since it was less than one day of
> work for the in tree ones. But if there is the desire for such an
> option I can try to add it. What should I use? An environment
> variable?
Hmm, I don't know LLVM's Makefile system well enough to know the
easiest way to implement an option; if it's non-trivial then maybe
it's not worth it.

I also don't know the workflow of most people doing out-of-tree work,
so I'm not sure how much impact this might have. It can obviously be
temporarily reverted locally pretty easily, but it assumes people are
paying attention to LLVMDev/llvm-commits and know what's going on.
Also, the commit causing all the new failures might not be as obvious
down the line to someone updating their tree irregularly (which is
probably true for a lot of academic LLVM users, I'm guessing.)

Just curious, does using pipefail give any information about where in
the pipe the failure actually comes from? Some kind of message would
be useful for debugging purposes, in addition to explaining what's
going on to someone who wasn't watching dev lists and commit messages
carefully.

Anyway, perhaps none of this is as big of a deal as I'm making it out
to be; I'll leave it to someone with more awareness of downstream
workflows to comment further.
>
> Cheers,
> Rafael
Stephen

> Hmm, I don't know LLVM's Makefile system well enough to know the
> easiest way to implement an option; if it's non-trivial then maybe
> it's not worth it.
That is my impression at least. These errors are somewhat easy to
introduce, but also easy to fix.
> I also don't know the workflow of most people doing out-of-tree work,
> so I'm not sure how much impact this might have. It can obviously be
> temporarily reverted locally pretty easily, but it assumes people are
> paying attention to LLVMDev/llvm-commits and know what's going on.
> Also, the commit causing all the new failures might not be as obvious
> down the line to someone updating their tree irregularly (which is
> probably true for a lot of academic LLVM users, I'm guessing.)
>
> Just curious, does using pipefail give any information about where in
> the pipe the failure actually comes from? Some kind of message would
> be useful for debugging purposes, in addition to explaining what's
> going on to someone who wasn't watching dev lists and commit messages
> carefully.
With the external (shell based) testing we don't get anything. With
the internal (python based) one we get the last failing command line.
It doesn't say which component of it failed, but the vast majority of
the pipes we have are of the form "command | FileCheck", so it is
clear that "command" must be the one failing since the pipe was
passing before enabling pipefail.

Cheers,
Rafael