llvm dev - Sep 2021 - Adding an execution result keyword in LIT

Hey all,

I am working on adding an execution result keyword such as `PASSED`,
`FAILED`, `XFAILED`, etc. to LIT to indicate the result for each RUN
execution. For example,

: 'RUN: at line 1' *(PASSED)*;
/home/anshil/amd-workspace/llvm-project/build/bin/llc -march=amdgcn
-mcpu=tahiti -verify-machineinstrs <
/home/anshil/amd-workspace/llvm-project/llvm/test/CodeGen/AMDGPU/inline-calls.ll
| /home/anshil/amd-workspace/llvm-project/build/bin/FileCheck
/home/anshil/amd-workspace/llvm-project/llvm/test/CodeGen/AMDGPU/inline-calls.ll

I was wondering what would be the most appropriate place to implement this?
I was thinking somewhere under
`parseIntegratedTestScriptsCommands(source_path, keywords)` in
TestRunner.py but I am open to suggestions.

Best,

Anshil Gandhi
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210920/10edc9d5/attachment.html>

Thanks! I've been wanting this annotation for a while now :)

I can help review something, but I wouldn't be able off-hand to make a
suggestion on how to do it!

On Mon, Sep 20, 2021 at 2:51 PM Anshil Gandhi via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> Hey all,
>
> I am working on adding an execution result keyword such as `PASSED`,
`FAILED`, `XFAILED`, etc. to LIT to indicate the result for each RUN execution.
For example,
>
> : 'RUN: at line 1' (PASSED);
/home/anshil/amd-workspace/llvm-project/build/bin/llc -march=amdgcn -mcpu=tahiti
-verify-machineinstrs <
/home/anshil/amd-workspace/llvm-project/llvm/test/CodeGen/AMDGPU/inline-calls.ll
| /home/anshil/amd-workspace/llvm-project/build/bin/FileCheck
/home/anshil/amd-workspace/llvm-project/llvm/test/CodeGen/AMDGPU/inline-calls.ll
>
> I was wondering what would be the most appropriate place to implement this?
I was thinking somewhere under `parseIntegratedTestScriptsCommands(source_path,
keywords)` in TestRunner.py but I am open to suggestions.
>
> Best,
>
> Anshil Gandhi
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

The "RUN: at line 1" part is printed by prepending an null command (a
colon) to the command line, which personally found to be a rather
strange implementation. Since PASSED/FAILED depends on the result, it
could at most be done by appending a command. Maybe both should be
printed in python itself, somewhere in TestRunner.py executeShCmd.
This executes all RUN-lines at once, with a single time-out. Something
that executes after each RUN line would need to be added. However,
printing the RUN-line before executing the command has the theoretical
advantage that one can could see where the current execution is stuck,
which may not matter that much since typically output is printed only
after it is known that the command failed.

Michael

Am Mo., 20. Sept. 2021 um 16:51 Uhr schrieb Anshil Gandhi via llvm-dev
<llvm-dev at lists.llvm.org>:
>
> Hey all,
>
> I am working on adding an execution result keyword such as `PASSED`,
`FAILED`, `XFAILED`, etc. to LIT to indicate the result for each RUN execution.
For example,
>
> : 'RUN: at line 1' (PASSED);
/home/anshil/amd-workspace/llvm-project/build/bin/llc -march=amdgcn -mcpu=tahiti
-verify-machineinstrs <
/home/anshil/amd-workspace/llvm-project/llvm/test/CodeGen/AMDGPU/inline-calls.ll
| /home/anshil/amd-workspace/llvm-project/build/bin/FileCheck
/home/anshil/amd-workspace/llvm-project/llvm/test/CodeGen/AMDGPU/inline-calls.ll
>
> I was wondering what would be the most appropriate place to implement this?
I was thinking somewhere under `parseIntegratedTestScriptsCommands(source_path,
keywords)` in TestRunner.py but I am open to suggestions.
>
> Best,
>
> Anshil Gandhi
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev