Sedat Dilek via llvm-dev
2018-May-07 10:01 UTC
[llvm-dev] [clang] Running a single testcase
On Mon, May 7, 2018 at 5:52 AM, Brian Cain <brian.cain at gmail.com> wrote:> The simplest way to run a clang test case that I know of is to clone both > llvm and clang repos, run all the tests, then run an individual test. > > IIRC like so: > > git clone llvm ...... > cd llvm/tools > git clone clang ..... > cd ../../ > mkdir build > cd build > cmake ../llvm > ninja check-clang > ./bin/llvm-lit -v ./tools/clang/test/Sema/asm.c >Hi Brian, The cmake-line should be... # apt-get install ninja-build $ cmake -G Ninja ../llvm Unfortunately, on the latest Git my build fails which gcc-7 (see attached file for details). Do I really need to build clang-from-git? I have llvm-7/clang-7 binaries installed on my system? But I run the test after symlinking clang-7 ---> clang... $ ./bin/llvm-lit -v ./tools/clang/test/Sema/asm.c llvm-lit: /home/sdi/src/llvm/llvm/utils/lit/lit/llvm/config.py:334: note: using clang: /usr/bin/clang llvm-lit: /home/sdi/src/llvm/llvm/utils/lit/lit/llvm/subst.py:126: note: Did not find c-index-test in /home/sdi/src/llvm/build/./bin:/home/sdi/src/llvm/build/./bin llvm-lit: /home/sdi/src/llvm/llvm/utils/lit/lit/llvm/subst.py:126: note: Did not find clang-check in /home/sdi/src/llvm/build/./bin:/home/sdi/src/llvm/build/./bin llvm-lit: /home/sdi/src/llvm/llvm/utils/lit/lit/llvm/subst.py:126: note: Did not find opt in /home/sdi/src/llvm/build/./bin:/home/sdi/src/llvm/build/./bin -- Testing: 1 tests, 1 threads -- PASS: Clang :: Sema/asm.c (1 of 1) Testing Time: 0.23s Expected Passes : 1 Looks good? Thanks. Regards, - Sedat -> > > On Sun, May 6, 2018 at 7:10 AM, Sedat Dilek via llvm-dev > <llvm-dev at lists.llvm.org> wrote: >> >> Hi, >> >> while experimenting with llvmlinux on Debian/testing AMD64 I wanted to >> run some x86-64 ASM tests. >> >> I fell over [1] and wanted to run it. >> >> So, I cloned clang from Git... >> >> $ git clone https://github.com/llvm-mirror/clang.git >> >> I looked through some docs where I have seen I need "llvm-lit" or >> "lit.py". >> The Debian package llvm-7-tools from <apt.llvm.org> does ship "lit.py". >> Furthermore, I have installed llvm-7 and clang-7 packages. >> The version is 1:7~svn330207-1~exp1+0~20180417201234.1709~1.gbp6fb10d. >> >> [5,6] have some examples on running a single testcase: >> >> For example: >> >> python C:\Tools\llvm\utils\lit\lit.py -sv >> --param=build_mode=Win32 --param=build_config=Debug >> --param=clang_site_config=C:\Tools\build\tools\clang\test\lit.site.cfg >> C:\Tools\llvm\tools\clang\test\Sema\wchar.c >> >> I transformed this... >> >> $ python /usr/lib/llvm-7/build/utils/lit/lit.py -sv >> /home/sdi/src/linux-kernel/clang/test/Sema/asm.c >> lit.py: /usr/lib/llvm-7/build/utils/lit/lit/TestingConfig.py:101: >> fatal: unable to parse config file >> '/home/sdi/src/linux-kernel/clang/test/lit.cfg.py', traceback: >> Traceback (most recent call last): >> File "/usr/lib/llvm-7/build/utils/lit/lit/TestingConfig.py", line >> 88, in load_from_path >> exec(compile(data, path, 'exec'), cfg_globals, None) >> File "/home/sdi/src/linux-kernel/clang/test/lit.cfg.py", line 25, in >> <module> >> config.test_format = lit.formats.ShTest(not llvm_config.use_lit_shell) >> AttributeError: 'NoneType' object has no attribute 'use_lit_shell' >> >> Yes, I have a lit.cfg.py. >> >> $ LC_ALL=C ll clang/test/lit.* >> -rw-r--r-- 1 sdi sdi 5979 Apr 25 12:14 clang/test/lit.cfg.py >> -rw-r--r-- 1 sdi sdi 1906 Apr 25 12:14 clang/test/lit.site.cfg.py.in >> >> How can I create a lit.site.cfg.py to pass it as >> --param=clang_site_config=/path/to/lit.site.cfg.py? >> >> I have attached TestingConfig.py from llvm-7-tools and lit.cfg.py and >> lit.site.cfg.py.in from clang-git. >> >> What am I missing? >> What do I need to run a single clang testcase? >> >> Thanks in advance. >> >> Regards, >> - Sedat - >> >> [1] https://github.com/llvm-mirror/clang/raw/master/test/Sema/asm.c >> [2] https://llvm.org/docs/CommandGuide/lit.html >> [3] https://llvm.org/docs/TestingGuide.html >> [4] https://llvm.org/docs/TestSuiteMakefileGuide.html >> [5] http://clang.llvm.org/hacking.html >> [6] http://clang.llvm.org/hacking.html#testingCommands >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> > > > > -- > -Brian-------------- next part -------------- [ INSTRUCTIONS ] 8664 2018-05-07 10:00:08 git clone https://github.com/llvm-mirror/llvm.git 8665 2018-05-07 10:02:27 cd llvm/tools 8666 2018-05-07 10:02:37 git clone https://github.com/llvm-mirror/clang.git 8667 2018-05-07 10:04:41 cd ../../ 8668 2018-05-07 10:04:51 mkdir build 8669 2018-05-07 10:04:59 cd build 8670 2018-05-07 10:05:12 cmake ../llvm 8679 2018-05-07 10:19:43 cmake ../llvm clean 8682 2018-05-07 10:20:18 cd ../ 8683 2018-05-07 10:20:21 rm -rf build 8684 2018-05-07 10:20:24 mkdir build 8685 2018-05-07 10:20:26 cd build/ 8686 2018-05-07 10:20:30 cmake -G Ninja ../llvm 8687 2018-05-07 10:20:45 find ./ -name build.ninja 8688 2018-05-07 10:21:00 ninja check-clang 8689 2018-05-07 11:46:36 ./bin/llvm-lit -v ./tools/clang/test/Sema/asm.c 8690 2018-05-07 11:46:48 which clang 8691 2018-05-07 11:46:51 which clang-7 8692 2018-05-07 11:47:44 ./bin/llvm-lit -v ./tools/clang/test/Sema/asm.c [ BUILD-LOG ] sdi at iniza:~/src/llvm/build$ ninja check-clang [107/2954] Building CXX object lib/Support/CMakeFiles/LLVMSupport.dir/Process.cpp.o In file included from /home/sdi/src/llvm/llvm/lib/Support/Unix/Unix.h:24:0, from /home/sdi/src/llvm/llvm/lib/Support/Unix/Process.inc:14, from /home/sdi/src/llvm/llvm/lib/Support/Process.cpp:94: /home/sdi/src/llvm/llvm/include/llvm/Support/Errno.h:34:13: warning: mangled name for ‘decltype (F(llvm::sys::RetryAfterSignal::As ...)) llvm::sys::RetryAfterSignal(const FailT&, const Fun&, const Args& ...) [with FailT = int; Fun = int(int, stat*) throw (); Args = {int, stat*}]’ will change in C++17 because the exception specification is part of a function type [-Wnoexcept-type] inline auto RetryAfterSignal(const FailT &Fail, const Fun &F, ^~~~~~~~~~~~~~~~ [462/2954] Building CXX object lib/CodeGen/MIRParser/CMakeFiles/LLVMMIRParser.dir/MIParser.cpp.o In file included from /home/sdi/src/llvm/llvm/include/llvm/ADT/StringRef.h:13:0, from /home/sdi/src/llvm/llvm/include/llvm/ADT/StringMap.h:17, from /home/sdi/src/llvm/llvm/include/llvm/Support/Host.h:17, from /home/sdi/src/llvm/llvm/include/llvm/ADT/Hashing.h:49, from /home/sdi/src/llvm/llvm/include/llvm/ADT/ArrayRef.h:13, from /home/sdi/src/llvm/llvm/include/llvm/ADT/DenseMapInfo.h:17, from /home/sdi/src/llvm/llvm/include/llvm/ADT/DenseMap.h:17, from /home/sdi/src/llvm/llvm/lib/CodeGen/MIRParser/MIParser.h:17, from /home/sdi/src/llvm/llvm/lib/CodeGen/MIRParser/MIParser.cpp:14: /home/sdi/src/llvm/llvm/include/llvm/ADT/STLExtras.h:908:6: warning: mangled name for ‘bool llvm::all_of(R&&, UnaryPredicate) [with R = llvm::StringRef&; UnaryPredicate = int (*)(int) throw ()]’ will change in C++17 because the exception specification is part of a function type [-Wnoexcept-type] bool all_of(R &&Range, UnaryPredicate P) { ^~~~~~ [1440/2954] Building CXX object lib/Target/Hexagon/AsmParser/CMakeFiles/LLVMHexagonAsmParser.dir/HexagonAsmParser.cpp.o In file included from /home/sdi/src/llvm/llvm/include/llvm/ADT/StringRef.h:13:0, from /home/sdi/src/llvm/llvm/include/llvm/ADT/StringMap.h:17, from /home/sdi/src/llvm/llvm/include/llvm/Support/Host.h:17, from /home/sdi/src/llvm/llvm/include/llvm/ADT/Hashing.h:49, from /home/sdi/src/llvm/llvm/include/llvm/ADT/ArrayRef.h:13, from /home/sdi/src/llvm/llvm/include/llvm/Support/CommandLine.h:23, from /home/sdi/src/llvm/llvm/lib/Target/Hexagon/MCTargetDesc/HexagonMCTargetDesc.h:17, from /home/sdi/src/llvm/llvm/lib/Target/Hexagon/Hexagon.h:45, from /home/sdi/src/llvm/llvm/lib/Target/Hexagon/AsmParser/HexagonAsmParser.cpp:12: /home/sdi/src/llvm/llvm/include/llvm/ADT/STLExtras.h:948:6: warning: mangled name for ‘decltype (llvm::adl_begin(Range)) llvm::remove_if(R&&, UnaryPredicate) [with R = std::__cxx11::basic_string<char>&; UnaryPredicate = int (*)(int) throw ()]’ will change in C++17 because the exception specification is part of a function type [-Wnoexcept-type] auto remove_if(R &&Range, UnaryPredicate P) -> decltype(adl_begin(Range)) { ^~~~~~~~~ [1769/2954] Building CXX object lib/AsmParser/CMakeFiles/LLVMAsmParser.dir/LLLexer.cpp.o In file included from /home/sdi/src/llvm/llvm/include/llvm/ADT/StringRef.h:13:0, from /home/sdi/src/llvm/llvm/include/llvm/ADT/StringMap.h:17, from /home/sdi/src/llvm/llvm/include/llvm/Support/Host.h:17, from /home/sdi/src/llvm/llvm/include/llvm/ADT/Hashing.h:49, from /home/sdi/src/llvm/llvm/include/llvm/ADT/ArrayRef.h:13, from /home/sdi/src/llvm/llvm/include/llvm/ADT/APFloat.h:21, from /home/sdi/src/llvm/llvm/lib/AsmParser/LLLexer.h:18, from /home/sdi/src/llvm/llvm/lib/AsmParser/LLLexer.cpp:14: /home/sdi/src/llvm/llvm/include/llvm/ADT/STLExtras.h:908:6: warning: mangled name for ‘bool llvm::all_of(R&&, UnaryPredicate) [with R = llvm::StringRef&; UnaryPredicate = int (*)(int) throw ()]’ will change in C++17 because the exception specification is part of a function type [-Wnoexcept-type] bool all_of(R &&Range, UnaryPredicate P) { ^~~~~~ [2745/2954] Linking CXX executable bin/clang-import-test FAILED: bin/clang-import-test : && /usr/bin/c++ -fPIC -fvisibility-inlines-hidden -Werror=date-time -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -fno-common -Woverloaded-virtual -fno-strict-aliasing -g -Wl,-allow-shlib-undefined -Wl,-rpath-link,/home/sdi/src/llvm/build/./lib tools/clang/tools/clang-import-test/CMakeFiles/clang-import-test.dir/clang-import-test.cpp.o -o bin/clang-import-test -Wl,-rpath,"\$ORIGIN/../lib" lib/libLLVMCore.a lib/libLLVMSupport.a -lpthread lib/libclangAST.a lib/libclangBasic.a lib/libclangCodeGen.a lib/libclangDriver.a lib/libclangFrontend.a lib/libclangLex.a lib/libclangParse.a lib/libclangDriver.a lib/libLLVMOption.a lib/libclangSerialization.a lib/libclangSema.a lib/libclangEdit.a lib/libclangAnalysis.a lib/libclangAST.a lib/libclangLex.a lib/libclangBasic.a lib/libLLVMCoroutines.a lib/libLLVMCoverage.a lib/libLLVMLTO.a lib/libLLVMObjCARCOpts.a lib/libLLVMPasses.a lib/libLLVMipo.a lib/libLLVMIRReader.a lib/libLLVMAsmParser.a lib/libLLVMLinker.a lib/libLLVMInstrumentation.a lib/libLLVMVectorize.a lib/libLLVMCodeGen.a lib/libLLVMBitWriter.a lib/libLLVMScalarOpts.a lib/libLLVMAggressiveInstCombine.a lib/libLLVMInstCombine.a lib/libLLVMTarget.a lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a lib/libLLVMObject.a lib/libLLVMBitReader.a lib/libLLVMMCParser.a lib/libLLVMMC.a lib/libLLVMDebugInfoCodeView.a lib/libLLVMDebugInfoMSF.a lib/libLLVMProfileData.a lib/libLLVMCore.a lib/libLLVMBinaryFormat.a lib/libLLVMSupport.a -lz -lrt -ldl -ltinfo -lpthread -lm lib/libLLVMDemangle.a && : collect2: fatal error: ld terminated with signal 9 [Getötet] compilation terminated. [2750/2954] Linking CXX executable bin/clang-7 ninja: build stopped: subcommand failed. -sdi // 07-MAY-2018
Amara Emerson via llvm-dev
2018-May-07 14:03 UTC
[llvm-dev] [clang] Running a single testcase
> On 7 May 2018, at 11:01, Sedat Dilek via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > On Mon, May 7, 2018 at 5:52 AM, Brian Cain <brian.cain at gmail.com <mailto:brian.cain at gmail.com>> wrote: >> The simplest way to run a clang test case that I know of is to clone both >> llvm and clang repos, run all the tests, then run an individual test. >> >> IIRC like so: >> >> git clone llvm ...... >> cd llvm/tools >> git clone clang ..... >> cd ../../ >> mkdir build >> cd build >> cmake ../llvm >> ninja check-clang >> ./bin/llvm-lit -v ./tools/clang/test/Sema/asm.c >> > > Hi Brian, > > The cmake-line should be... > > # apt-get install ninja-build > $ cmake -G Ninja ../llvm > > Unfortunately, on the latest Git my build fails which gcc-7 (see > attached file for details). > Do I really need to build clang-from-git? > I have llvm-7/clang-7 binaries installed on my system? > > But I run the test after symlinking clang-7 ---> clang... > > $ ./bin/llvm-lit -v ./tools/clang/test/Sema/asm.c > llvm-lit: /home/sdi/src/llvm/llvm/utils/lit/lit/llvm/config.py:334: > note: using clang: /usr/bin/clang > llvm-lit: /home/sdi/src/llvm/llvm/utils/lit/lit/llvm/subst.py:126: > note: Did not find c-index-test in > /home/sdi/src/llvm/build/./bin:/home/sdi/src/llvm/build/./bin > llvm-lit: /home/sdi/src/llvm/llvm/utils/lit/lit/llvm/subst.py:126: > note: Did not find clang-check in > /home/sdi/src/llvm/build/./bin:/home/sdi/src/llvm/build/./bin > llvm-lit: /home/sdi/src/llvm/llvm/utils/lit/lit/llvm/subst.py:126: > note: Did not find opt in > /home/sdi/src/llvm/build/./bin:/home/sdi/src/llvm/build/./binThat looks like your system ran out of memory during linking, which on debug builds is possible especially if you use parallel jobs with -j. You can’t use a different clang binary to run tests and expect them to work, the tests are specific to each individual version of the compiler. Try using -DCMAKE_BUILD_TYPE=Release which requires less memory to link, unless you really need debug info in the binary, in which case try running ninja -j1 to avoid parallel jobs. Amara -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180507/27699945/attachment-0001.html>
Sedat Dilek via llvm-dev
2018-May-07 15:13 UTC
[llvm-dev] [clang] Running a single testcase
On Mon, May 7, 2018 at 4:03 PM, Amara Emerson <aemerson at apple.com> wrote:> On 7 May 2018, at 11:01, Sedat Dilek via llvm-dev <llvm-dev at lists.llvm.org> > wrote: > > On Mon, May 7, 2018 at 5:52 AM, Brian Cain <brian.cain at gmail.com> wrote: > > The simplest way to run a clang test case that I know of is to clone both > llvm and clang repos, run all the tests, then run an individual test. > > IIRC like so: > > git clone llvm ...... > cd llvm/tools > git clone clang ..... > cd ../../ > mkdir build > cd build > cmake ../llvm > ninja check-clang > ./bin/llvm-lit -v ./tools/clang/test/Sema/asm.c > > > Hi Brian, > > The cmake-line should be... > > # apt-get install ninja-build > $ cmake -G Ninja ../llvm > > Unfortunately, on the latest Git my build fails which gcc-7 (see > attached file for details). > Do I really need to build clang-from-git? > I have llvm-7/clang-7 binaries installed on my system? > > But I run the test after symlinking clang-7 ---> clang... > > $ ./bin/llvm-lit -v ./tools/clang/test/Sema/asm.c > llvm-lit: /home/sdi/src/llvm/llvm/utils/lit/lit/llvm/config.py:334: > note: using clang: /usr/bin/clang > llvm-lit: /home/sdi/src/llvm/llvm/utils/lit/lit/llvm/subst.py:126: > note: Did not find c-index-test in > /home/sdi/src/llvm/build/./bin:/home/sdi/src/llvm/build/./bin > llvm-lit: /home/sdi/src/llvm/llvm/utils/lit/lit/llvm/subst.py:126: > note: Did not find clang-check in > /home/sdi/src/llvm/build/./bin:/home/sdi/src/llvm/build/./bin > llvm-lit: /home/sdi/src/llvm/llvm/utils/lit/lit/llvm/subst.py:126: > note: Did not find opt in > /home/sdi/src/llvm/build/./bin:/home/sdi/src/llvm/build/./bin > > > > That looks like your system ran out of memory during linking, which on debug > builds is possible especially if you use parallel jobs with -j. You can’t > use a different clang binary to run tests and expect them to work, the tests > are specific to each individual version of the compiler. > > Try using -DCMAKE_BUILD_TYPE=Release which requires less memory to link, > unless you really need debug info in the binary, in which case try running > ninja -j1 to avoid parallel jobs. >Hi Amara, yes, that sounds reasonable. I looked into my old build_llvm-toolchain.sh script and adapted it to build only the necessary stuff (see below). First, I thought I can reduce build-time by... sdi at iniza:~/src/llvm-toolchain/build$ ninja check-llvm-tools-llvm-lit [1808/1809] Running lit suite /home/sdi/src/llvm-toolchain/llvm/test/tools/llvm-lit Testing Time: 0.11s Expected Failures : 1 The test runs with some warnings... sdi at iniza:~/src/llvm-toolchain/build$ ./bin/llvm-lit -v ./tools/clang/test/Sema/asm.c llvm-lit: /home/sdi/src/llvm-toolchain/llvm/utils/lit/lit/llvm/config.py:334: note: using clang: /usr/bin/clang llvm-lit: /home/sdi/src/llvm-toolchain/llvm/utils/lit/lit/llvm/subst.py:126: note: Did not find c-index-test in /home/sdi/src/llvm-toolchain/build/./bin:/home/sdi/src/llvm-toolchain/build/./bin llvm-lit: /home/sdi/src/llvm-toolchain/llvm/utils/lit/lit/llvm/subst.py:126: note: Did not find clang-check in /home/sdi/src/llvm-toolchain/build/./bin:/home/sdi/src/llvm-toolchain/build/./bin llvm-lit: /home/sdi/src/llvm-toolchain/llvm/utils/lit/lit/llvm/subst.py:126: note: Did not find clang-diff in /home/sdi/src/llvm-toolchain/build/./bin:/home/sdi/src/llvm-toolchain/build/./bin llvm-lit: /home/sdi/src/llvm-toolchain/llvm/utils/lit/lit/llvm/subst.py:126: note: Did not find clang-format in /home/sdi/src/llvm-toolchain/build/./bin:/home/sdi/src/llvm-toolchain/build/./bin -- Testing: 1 tests, 1 threads -- PASS: Clang :: Sema/asm.c (1 of 1) Testing Time: 0.11s Expected Passes : 1 ... but it looks like stuff is missing, so I did... sdi at iniza:~/src/llvm-toolchain/build$ ninja check-clang [1012/1013] Running the Clang regression tests llvm-lit: /home/sdi/src/llvm-toolchain/llvm/utils/lit/lit/llvm/config.py:334: note: using clang: /home/sdi/src/llvm-toolchain/build/bin/clang Testing Time: 200.66s Expected Passes : 12028 Expected Failures : 18 Unsupported Tests : 256 sdi at iniza:~/src/llvm-toolchain/build$ ./bin/llvm-lit -v ./tools/clang/test/Sema/asm.c llvm-lit: /home/sdi/src/llvm-toolchain/llvm/utils/lit/lit/llvm/config.py:334: note: using clang: /home/sdi/src/llvm-toolchain/build/bin/clang -- Testing: 1 tests, 1 threads -- PASS: Clang :: Sema/asm.c (1 of 1) Testing Time: 0.10s Expected Passes : 1 This now looks good to me. How can I force to use a different clang than "/home/sdi/src/llvm-toolchain/build/bin/clang"? I have no symlink to /usr/bin/clang. Thanks. Regards, - Sedat - P.S.: New build-script [build_clang-with-ninja.sh ] #!/bin/sh export LANG=C export LC_ALL=C cd build CMAKE="cmake" CMAKE_OPTS="-G Ninja -DCMAKE_C_COMPILER=clang-7 -DCMAKE_CXX_COMPILER=clang++-7 -DLLVM_PARALLEL_COMPILE_JOBS=2 -DLLVM_PARALLEL_LINK_JOBS=1 -DCMAKE_BUILD_TYPE=RELEASE -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_TARGETS_TO_BUILD=X86" $CMAKE ../llvm $CMAKE_OPTS ninja check-clang - EOF -