llvm dev - May 2018 - [clang] Running a single testcase

On Mon, May 7, 2018 at 4:03 PM, Amara Emerson <aemerson at apple.com>
wrote:
> On 7 May 2018, at 11:01, Sedat Dilek via llvm-dev <llvm-dev at
lists.llvm.org>
> wrote:
>
> On Mon, May 7, 2018 at 5:52 AM, Brian Cain <brian.cain at gmail.com>
wrote:
>
> The simplest way to run a clang test case that I know of is to clone both
> llvm and clang repos, run all the tests, then run an individual test.
>
> IIRC like so:
>
> git clone llvm ......
> cd llvm/tools
> git clone clang .....
> cd ../../
> mkdir build
> cd build
> cmake ../llvm
> ninja check-clang
> ./bin/llvm-lit -v ./tools/clang/test/Sema/asm.c
>
>
> Hi Brian,
>
> The cmake-line should be...
>
> # apt-get install ninja-build
> $ cmake -G Ninja ../llvm
>
> Unfortunately, on the latest Git my build fails which gcc-7 (see
> attached file for details).
> Do I really need to build clang-from-git?
> I have llvm-7/clang-7 binaries installed on my system?
>
> But I run the test after symlinking clang-7 ---> clang...
>
> $ ./bin/llvm-lit -v ./tools/clang/test/Sema/asm.c
> llvm-lit: /home/sdi/src/llvm/llvm/utils/lit/lit/llvm/config.py:334:
> note: using clang: /usr/bin/clang
> llvm-lit: /home/sdi/src/llvm/llvm/utils/lit/lit/llvm/subst.py:126:
> note: Did not find c-index-test in
> /home/sdi/src/llvm/build/./bin:/home/sdi/src/llvm/build/./bin
> llvm-lit: /home/sdi/src/llvm/llvm/utils/lit/lit/llvm/subst.py:126:
> note: Did not find clang-check in
> /home/sdi/src/llvm/build/./bin:/home/sdi/src/llvm/build/./bin
> llvm-lit: /home/sdi/src/llvm/llvm/utils/lit/lit/llvm/subst.py:126:
> note: Did not find opt in
> /home/sdi/src/llvm/build/./bin:/home/sdi/src/llvm/build/./bin
>
>
>
> That looks like your system ran out of memory during linking, which on
debug
> builds is possible especially if you use parallel jobs with -j. You can’t
> use a different clang binary to run tests and expect them to work, the
tests
> are specific to each individual version of the compiler.
>
> Try using -DCMAKE_BUILD_TYPE=Release which requires less memory to link,
> unless you really need debug info in the binary, in which case try running
> ninja -j1 to avoid parallel jobs.
>
Hi Amara,

yes, that sounds reasonable.

I looked into my old build_llvm-toolchain.sh script and adapted it to
build only the necessary stuff (see below).

First, I thought I can reduce build-time by...

sdi at iniza:~/src/llvm-toolchain/build$ ninja check-llvm-tools-llvm-lit
[1808/1809] Running lit suite
/home/sdi/src/llvm-toolchain/llvm/test/tools/llvm-lit
Testing Time: 0.11s
  Expected Failures  : 1

The test runs with some warnings...

sdi at iniza:~/src/llvm-toolchain/build$ ./bin/llvm-lit -v
./tools/clang/test/Sema/asm.c
llvm-lit: /home/sdi/src/llvm-toolchain/llvm/utils/lit/lit/llvm/config.py:334:
note: using clang: /usr/bin/clang
llvm-lit: /home/sdi/src/llvm-toolchain/llvm/utils/lit/lit/llvm/subst.py:126:
note: Did not find c-index-test in
/home/sdi/src/llvm-toolchain/build/./bin:/home/sdi/src/llvm-toolchain/build/./bin
llvm-lit: /home/sdi/src/llvm-toolchain/llvm/utils/lit/lit/llvm/subst.py:126:
note: Did not find clang-check in
/home/sdi/src/llvm-toolchain/build/./bin:/home/sdi/src/llvm-toolchain/build/./bin
llvm-lit: /home/sdi/src/llvm-toolchain/llvm/utils/lit/lit/llvm/subst.py:126:
note: Did not find clang-diff in
/home/sdi/src/llvm-toolchain/build/./bin:/home/sdi/src/llvm-toolchain/build/./bin
llvm-lit: /home/sdi/src/llvm-toolchain/llvm/utils/lit/lit/llvm/subst.py:126:
note: Did not find clang-format in
/home/sdi/src/llvm-toolchain/build/./bin:/home/sdi/src/llvm-toolchain/build/./bin
-- Testing: 1 tests, 1 threads --
PASS: Clang :: Sema/asm.c (1 of 1)
Testing Time: 0.11s
  Expected Passes    : 1

... but it looks like stuff is missing, so I did...

sdi at iniza:~/src/llvm-toolchain/build$ ninja check-clang
[1012/1013] Running the Clang regression tests
llvm-lit: /home/sdi/src/llvm-toolchain/llvm/utils/lit/lit/llvm/config.py:334:
note: using clang: /home/sdi/src/llvm-toolchain/build/bin/clang
Testing Time: 200.66s
  Expected Passes    : 12028
  Expected Failures  : 18
  Unsupported Tests  : 256

sdi at iniza:~/src/llvm-toolchain/build$ ./bin/llvm-lit -v
./tools/clang/test/Sema/asm.c
llvm-lit: /home/sdi/src/llvm-toolchain/llvm/utils/lit/lit/llvm/config.py:334:
note: using clang: /home/sdi/src/llvm-toolchain/build/bin/clang
-- Testing: 1 tests, 1 threads --
PASS: Clang :: Sema/asm.c (1 of 1)
Testing Time: 0.10s
  Expected Passes    : 1

This now looks good to me.

How can I force to use a different clang than
"/home/sdi/src/llvm-toolchain/build/bin/clang"?
I have no symlink to /usr/bin/clang.

Thanks.

Regards,
- Sedat -

P.S.: New build-script

[build_clang-with-ninja.sh  ]

#!/bin/sh

export LANG=C
export LC_ALL=C

cd build

CMAKE="cmake"

CMAKE_OPTS="-G Ninja -DCMAKE_C_COMPILER=clang-7
-DCMAKE_CXX_COMPILER=clang++-7 -DLLVM_PARALLEL_COMPILE_JOBS=2
-DLLVM_PARALLEL_LINK_JOBS=1 -DCMAKE_BUILD_TYPE=RELEASE
-DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_TARGETS_TO_BUILD=X86"

$CMAKE ../llvm $CMAKE_OPTS

ninja check-clang
- EOF -

On Mon, May 7, 2018 at 10:13 AM, Sedat Dilek <sedat.dilek at gmail.com>
wrote:
> ...
>
> sdi at iniza:~/src/llvm-toolchain/build$ ./bin/llvm-lit -v
> ./tools/clang/test/Sema/asm.c
> llvm-lit: /home/sdi/src/llvm-toolchain/llvm/utils/lit/lit/llvm/
> config.py:334:
> note: using clang: /home/sdi/src/llvm-toolchain/build/bin/clang
> -- Testing: 1 tests, 1 threads --
> PASS: Clang :: Sema/asm.c (1 of 1)
> Testing Time: 0.10s
>   Expected Passes    : 1
>
> This now looks good to me.
>
> How can I force to use a different clang than
> "/home/sdi/src/llvm-toolchain/build/bin/clang"?
> I have no symlink to /usr/bin/clang.
>
>
 Can you give a little more context?  What is your ultimate goal for this
activity?  Why would you want to run the test in this fashion and what
meaning would you attribute to the test passing or failing?

-Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180507/31ccd309/attachment.html>

On Mon, May 7, 2018 at 5:19 PM, Brian Cain <brian.cain at gmail.com>
wrote:
>
>
> On Mon, May 7, 2018 at 10:13 AM, Sedat Dilek <sedat.dilek at
gmail.com> wrote:
>>
>> ...
>
>
>>
>> sdi at iniza:~/src/llvm-toolchain/build$ ./bin/llvm-lit -v
>> ./tools/clang/test/Sema/asm.c
>> llvm-lit:
>> /home/sdi/src/llvm-toolchain/llvm/utils/lit/lit/llvm/config.py:334:
>> note: using clang: /home/sdi/src/llvm-toolchain/build/bin/clang
>> -- Testing: 1 tests, 1 threads --
>> PASS: Clang :: Sema/asm.c (1 of 1)
>> Testing Time: 0.10s
>>   Expected Passes    : 1
>>
>> This now looks good to me.
>>
>> How can I force to use a different clang than
>> "/home/sdi/src/llvm-toolchain/build/bin/clang"?
>> I have no symlink to /usr/bin/clang.
>>
>
>  Can you give a little more context?  What is your ultimate goal for this
> activity?  Why would you want to run the test in this fashion and what
> meaning would you attribute to the test passing or failing?
>
When compiling a Linux-kernel I hit a problem in the
KVM/paravirtualization area and "clobbering RCX" on Debian/testing
AMD64.
The question came up, is this is a compiler bug or not.
Details see [1].

Thus, I was looking for a reproducible testcase.
I have seen that clang ships some tests.
So, I was curious to run a single one.
Not sure if clang/test/Sema/asm.c is suitable here to see "clobbering
RCX" on Debian/x86-64.

On my host I have installed a clang-7 package from <apt.llvm.org>.
I wanted to test against that version.

Does this help you?

- Sedat -

[1] https://lkml.org/lkml/2018/4/24/1227