llvm dev - Sep 2021 - Fwd: Adding an execution result keyword in LIT

---------- Forwarded message ---------
From: Anshil Gandhi <gandhi21299 at gmail.com>
Date: Tue, Sep 21, 2021 at 11:38 PM
Subject: Re: [llvm-dev] Adding an execution result keyword in LIT
To: Michael Kruse <llvmdev at meinersbur.de>


Hey Michael,

Thanks for your response! executeShCmd() is never being called when I run
`./bin/llvm-lit -v <test-file>`.

I am thinking of somehow interacting with the subprocess.Popen(...) in
lit/util.py::executeCommand(...) since this is where the script is
executed. I need to obtain the execution result status somehow, this is the
tricky part.

Best,

Anshil

On Mon, Sep 20, 2021 at 6:06 PM Michael Kruse <llvmdev at meinersbur.de>
wrote:
> The "RUN: at line 1" part is printed by prepending an null
command (a
> colon) to the command line, which personally found to be a rather
> strange implementation. Since PASSED/FAILED depends on the result, it
> could at most be done by appending a command. Maybe both should be
> printed in python itself, somewhere in TestRunner.py executeShCmd.
> This executes all RUN-lines at once, with a single time-out. Something
> that executes after each RUN line would need to be added. However,
> printing the RUN-line before executing the command has the theoretical
> advantage that one can could see where the current execution is stuck,
> which may not matter that much since typically output is printed only
> after it is known that the command failed.
>
> Michael
>
> Am Mo., 20. Sept. 2021 um 16:51 Uhr schrieb Anshil Gandhi via llvm-dev
> <llvm-dev at lists.llvm.org>:
> >
> > Hey all,
> >
> > I am working on adding an execution result keyword such as `PASSED`,
> `FAILED`, `XFAILED`, etc. to LIT to indicate the result for each RUN
> execution. For example,
> >
> > : 'RUN: at line 1' (PASSED);
> /home/anshil/amd-workspace/llvm-project/build/bin/llc -march=amdgcn
> -mcpu=tahiti -verify-machineinstrs <
>
/home/anshil/amd-workspace/llvm-project/llvm/test/CodeGen/AMDGPU/inline-calls.ll
> | /home/anshil/amd-workspace/llvm-project/build/bin/FileCheck
>
/home/anshil/amd-workspace/llvm-project/llvm/test/CodeGen/AMDGPU/inline-calls.ll
> >
> > I was wondering what would be the most appropriate place to implement
> this? I was thinking somewhere under
> `parseIntegratedTestScriptsCommands(source_path, keywords)` in
> TestRunner.py but I am open to suggestions.
> >
> > Best,
> >
> > Anshil Gandhi
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210921/b6ad6712/attachment.html>

I wonder if I could rely on the exit status of each child process and
conclude PASSED or FAILED based on that. Please let me know if this is the
case.

Anshil

On Tue, Sep 21, 2021 at 11:38 PM Anshil Gandhi <gandhi21299 at gmail.com>
wrote:
>
>
> ---------- Forwarded message ---------
> From: Anshil Gandhi <gandhi21299 at gmail.com>
> Date: Tue, Sep 21, 2021 at 11:38 PM
> Subject: Re: [llvm-dev] Adding an execution result keyword in LIT
> To: Michael Kruse <llvmdev at meinersbur.de>
>
>
> Hey Michael,
>
> Thanks for your response! executeShCmd() is never being called when I run
> `./bin/llvm-lit -v <test-file>`.
>
> I am thinking of somehow interacting with the subprocess.Popen(...) in
> lit/util.py::executeCommand(...) since this is where the script is
> executed. I need to obtain the execution result status somehow, this is the
> tricky part.
>
> Best,
>
> Anshil
>
> On Mon, Sep 20, 2021 at 6:06 PM Michael Kruse <llvmdev at
meinersbur.de>
> wrote:
>
>> The "RUN: at line 1" part is printed by prepending an null
command (a
>> colon) to the command line, which personally found to be a rather
>> strange implementation. Since PASSED/FAILED depends on the result, it
>> could at most be done by appending a command. Maybe both should be
>> printed in python itself, somewhere in TestRunner.py executeShCmd.
>> This executes all RUN-lines at once, with a single time-out. Something
>> that executes after each RUN line would need to be added. However,
>> printing the RUN-line before executing the command has the theoretical
>> advantage that one can could see where the current execution is stuck,
>> which may not matter that much since typically output is printed only
>> after it is known that the command failed.
>>
>> Michael
>>
>> Am Mo., 20. Sept. 2021 um 16:51 Uhr schrieb Anshil Gandhi via llvm-dev
>> <llvm-dev at lists.llvm.org>:
>> >
>> > Hey all,
>> >
>> > I am working on adding an execution result keyword such as
`PASSED`,
>> `FAILED`, `XFAILED`, etc. to LIT to indicate the result for each RUN
>> execution. For example,
>> >
>> > : 'RUN: at line 1' (PASSED);
>> /home/anshil/amd-workspace/llvm-project/build/bin/llc -march=amdgcn
>> -mcpu=tahiti -verify-machineinstrs <
>>
/home/anshil/amd-workspace/llvm-project/llvm/test/CodeGen/AMDGPU/inline-calls.ll
>> | /home/anshil/amd-workspace/llvm-project/build/bin/FileCheck
>>
/home/anshil/amd-workspace/llvm-project/llvm/test/CodeGen/AMDGPU/inline-calls.ll
>> >
>> > I was wondering what would be the most appropriate place to
implement
>> this? I was thinking somewhere under
>> `parseIntegratedTestScriptsCommands(source_path, keywords)` in
>> TestRunner.py but I am open to suggestions.
>> >
>> > Best,
>> >
>> > Anshil Gandhi
>> > _______________________________________________
>> > LLVM Developers mailing list
>> > llvm-dev at lists.llvm.org
>> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210921/23a6da0f/attachment.html>