hi,all: i really want to how to write a regression test case,but i do not find out a entire document about it. By reading LLVM Testing Infrastructure Guide and FileCheck - Flexible pattern matching file verifier, i only have a idea,but it is not enough for me to write a test case.i need more detail about how to write a RUN:lines. thanks for you scan,wish your letter. best wishes, changcheng
Hi, changcheng, There are following steps to be taken to write a test case: 1. create a directory in test, say "test/XXX" 2. create a file named lit.local.cfg, which is a configuration file. The straightforward way is to copy an existed one, like "test/CodeGen/SPARC/lit.local.cfg". You might need make some modification on it. 3. create a file, say "example.ll" (The name of file does not matter, but the suffix of the file name DOES matter. See config.suffixes = [ ... ] in lit.local.cfg) for more details. example.ll includes normal content of LLVM IR, and the ";RUN ..." directives at the beginning of the file. 4. run all regression test cases with "make check", which includes above test case newly added. Or you might want to run just that case with "llvm-lit test/XXX" or "llvm-lit test/XXX/example.ll" llvm-lit resides in BUILD tree, distinguished from source tree and install tree. You can find it under your-build-tree/Release+Asserts/bin/ or your-build-tree/Debug+Asserts/bin/ or likewise, which depends on your build configuration. By the way, in the following ";RUN ..." directive ; RUN: llc < %s -march=x86 | FileCheck %s %s indicates current file under test. I guess the first %s stands for the input for llc, while the second and last %s specifies what to check using "; CHECK ..." directives since you write them down on exactly CURRENT file. Hope it work for you. Regards. 2012/8/20 Changcheng Wang <changcheng at multicorewareinc.com>:> hi,all: > i really want to how to write a regression test case,but i do not find > out a entire document about it. > By reading LLVM Testing Infrastructure Guide and FileCheck - Flexible > pattern matching file verifier, i only have a idea,but it is not > enough for me to write a test case.i need more detail about how to > write a RUN:lines. > > thanks for you scan,wish your letter. > > best wishes, > > changcheng > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev-- 杨勇勇 (Yang Yongyong)
2012/8/21 Changcheng Wang <changcheng at multicorewareinc.com>:> Hi,Yang > thanks for your entire answer,i will do it follow you. > still another question puzzled me:i write a hello.c file like this: > > //RUN: llc -march=c < %s | FileCheck %s > #include"stdio.h" > > int main() { > printf ("Helloworld.\n"); > return 0; > } > > in fact, i want to thanslate it to a hello.ll file like this: > ;RUN: llc -march=c < %s | FileCheck %s > ....... > ;CHECK..... > > but,when i thanslate the hello.c with clang,the sentence "//RUN: llc > -march=c < %s | FileCheck %s" was treated as comment. > > it samed the method translating hello.c to hello.ll is wrong,isnot it?To translate a .c file to a .ll file, you should wirte, for example, clang -emit-llvm -S test.c -o test.ll And the corresponding command for regression test might be: ; RUN: clang < %s -emit-llvm -S | FileCheck %s Regards. (PS. When you reply to this email, make sure you send a copy to llvmdev at cs.uiuc.edu)> > thanks very much! > best wishes > > changcheng > > On Tue, Aug 21, 2012 at 10:46 AM, Triple Yang <triple.yang at gmail.com> wrote: >> Hi, changcheng, >> >> There are following steps to be taken to write a test case: >> 1. create a directory in test, say "test/XXX" >> 2. create a file named lit.local.cfg, which is a configuration file. >> The straightforward way is to copy an existed one, >> like "test/CodeGen/SPARC/lit.local.cfg". You might need make some >> modification on it. >> 3. create a file, say "example.ll" >> (The name of file does not matter, but the suffix of the file name >> DOES matter. See config.suffixes = [ ... ] in lit.local.cfg) >> for more details. >> example.ll includes normal content of LLVM IR, and the ";RUN ..." >> directives at the beginning of the file. >> 4. run all regression test cases with "make check", which includes >> above test case newly added. >> Or you might want to run just that case with "llvm-lit test/XXX" >> or "llvm-lit test/XXX/example.ll" >> llvm-lit resides in BUILD tree, distinguished from source tree and >> install tree. You can find it under >> your-build-tree/Release+Asserts/bin/ or >> your-build-tree/Debug+Asserts/bin/ or likewise, which depends on your >> build >> configuration. >> >> By the way, in the following ";RUN ..." directive >> ; RUN: llc < %s -march=x86 | FileCheck %s >> %s indicates current file under test. I guess the first %s stands for >> the input for llc, while the second and last %s specifies >> what to check using "; CHECK ..." directives since you write them down >> on exactly CURRENT file. >> >> Hope it work for you. >> Regards. >> >> 2012/8/20 Changcheng Wang <changcheng at multicorewareinc.com>: >>> hi,all: >>> i really want to how to write a regression test case,but i do not find >>> out a entire document about it. >>> By reading LLVM Testing Infrastructure Guide and FileCheck - Flexible >>> pattern matching file verifier, i only have a idea,but it is not >>> enough for me to write a test case.i need more detail about how to >>> write a RUN:lines. >>> >>> thanks for you scan,wish your letter. >>> >>> best wishes, >>> >>> changcheng >>> _______________________________________________ >>> LLVM Developers mailing list >>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >> >> >> >> -- >> 杨勇勇 (Yang Yongyong)-- 杨勇勇 (Yang Yongyong)
2012/8/21 Triple Yang <triple.yang at gmail.com>:> 2012/8/21 Changcheng Wang <changcheng at multicorewareinc.com>: >> Hi,Yang >> thanks for your entire answer,i will do it follow you. >> still another question puzzled me:i write a hello.c file like this: >> >> //RUN: llc -march=c < %s | FileCheck %s >> #include"stdio.h" >> >> int main() { >> printf ("Helloworld.\n"); >> return 0; >> } >> >> in fact, i want to thanslate it to a hello.ll file like this: >> ;RUN: llc -march=c < %s | FileCheck %s >> ....... >> ;CHECK..... >> >> but,when i thanslate the hello.c with clang,the sentence "//RUN: llc >> -march=c < %s | FileCheck %s" was treated as comment. >> >> it samed the method translating hello.c to hello.ll is wrong,isnot it? > > To translate a .c file to a .ll file, you should wirte, for example, > clang -emit-llvm -S test.c -o test.ll > > And the corresponding command for regression test might be: > ; RUN: clang < %s -emit-llvm -S | FileCheck %s >Sorry for my carelessness. The method mentioned above does not work at all. An alternative way is as follows: First compile .c files into .ll files, then modify .ll files to do regression test. You can modify the Makefile under current directory.> Regards. > > (PS. When you reply to this email, make sure you send a copy to > llvmdev at cs.uiuc.edu) > >> >> thanks very much! >> best wishes >> >> changcheng >> >> On Tue, Aug 21, 2012 at 10:46 AM, Triple Yang <triple.yang at gmail.com> wrote: >>> Hi, changcheng, >>> >>> There are following steps to be taken to write a test case: >>> 1. create a directory in test, say "test/XXX" >>> 2. create a file named lit.local.cfg, which is a configuration file. >>> The straightforward way is to copy an existed one, >>> like "test/CodeGen/SPARC/lit.local.cfg". You might need make some >>> modification on it. >>> 3. create a file, say "example.ll" >>> (The name of file does not matter, but the suffix of the file name >>> DOES matter. See config.suffixes = [ ... ] in lit.local.cfg) >>> for more details. >>> example.ll includes normal content of LLVM IR, and the ";RUN ..." >>> directives at the beginning of the file. >>> 4. run all regression test cases with "make check", which includes >>> above test case newly added. >>> Or you might want to run just that case with "llvm-lit test/XXX" >>> or "llvm-lit test/XXX/example.ll" >>> llvm-lit resides in BUILD tree, distinguished from source tree and >>> install tree. You can find it under >>> your-build-tree/Release+Asserts/bin/ or >>> your-build-tree/Debug+Asserts/bin/ or likewise, which depends on your >>> build >>> configuration. >>> >>> By the way, in the following ";RUN ..." directive >>> ; RUN: llc < %s -march=x86 | FileCheck %s >>> %s indicates current file under test. I guess the first %s stands for >>> the input for llc, while the second and last %s specifies >>> what to check using "; CHECK ..." directives since you write them down >>> on exactly CURRENT file. >>> >>> Hope it work for you. >>> Regards. >>> >>> 2012/8/20 Changcheng Wang <changcheng at multicorewareinc.com>: >>>> hi,all: >>>> i really want to how to write a regression test case,but i do not find >>>> out a entire document about it. >>>> By reading LLVM Testing Infrastructure Guide and FileCheck - Flexible >>>> pattern matching file verifier, i only have a idea,but it is not >>>> enough for me to write a test case.i need more detail about how to >>>> write a RUN:lines. >>>> >>>> thanks for you scan,wish your letter. >>>> >>>> best wishes, >>>> >>>> changcheng >>>> _______________________________________________ >>>> LLVM Developers mailing list >>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >>> >>> >>> >>> -- >>> 杨勇勇 (Yang Yongyong) > > > > -- > 杨勇勇 (Yang Yongyong)-- 杨勇勇 (Yang Yongyong)
2012/8/21 Triple Yang <triple.yang at gmail.com>:> 2012/8/21 Changcheng Wang <changcheng at multicorewareinc.com>: >> Hi,Yang >> thanks for your entire answer,i will do it follow you. >> still another question puzzled me:i write a hello.c file like this: >> >> //RUN: llc -march=c < %s | FileCheck %s >> #include"stdio.h" >> >> int main() { >> printf ("Helloworld.\n"); >> return 0; >> } >> >> in fact, i want to thanslate it to a hello.ll file like this: >> ;RUN: llc -march=c < %s | FileCheck %s >> ....... >> ;CHECK..... >> >> but,when i thanslate the hello.c with clang,the sentence "//RUN: llc >> -march=c < %s | FileCheck %s" was treated as comment. >> >> it samed the method translating hello.c to hello.ll is wrong,isnot it? > > To translate a .c file to a .ll file, you should wirte, for example, > clang -emit-llvm -S test.c -o test.ll > > And the corresponding command for regression test might be: > ; RUN: clang < %s -emit-llvm -S | FileCheck %sWe can write: // RUN: clang < %s -emit-llvm -S in a .c file to test clang. And if we use FileCheck, like: // RUN: clang < %s -emit-llvm -S | FileCheck %s make sure there is some "// CHECK:" in that file, or lit will report a failure. See utils/lit/lit/ExampleTests/Clang for more details.> > Regards. > > (PS. When you reply to this email, make sure you send a copy to > llvmdev at cs.uiuc.edu) > >> >> thanks very much! >> best wishes >> >> changcheng >> >> On Tue, Aug 21, 2012 at 10:46 AM, Triple Yang <triple.yang at gmail.com> wrote: >>> Hi, changcheng, >>> >>> There are following steps to be taken to write a test case: >>> 1. create a directory in test, say "test/XXX" >>> 2. create a file named lit.local.cfg, which is a configuration file. >>> The straightforward way is to copy an existed one, >>> like "test/CodeGen/SPARC/lit.local.cfg". You might need make some >>> modification on it. >>> 3. create a file, say "example.ll" >>> (The name of file does not matter, but the suffix of the file name >>> DOES matter. See config.suffixes = [ ... ] in lit.local.cfg) >>> for more details. >>> example.ll includes normal content of LLVM IR, and the ";RUN ..." >>> directives at the beginning of the file. >>> 4. run all regression test cases with "make check", which includes >>> above test case newly added. >>> Or you might want to run just that case with "llvm-lit test/XXX" >>> or "llvm-lit test/XXX/example.ll" >>> llvm-lit resides in BUILD tree, distinguished from source tree and >>> install tree. You can find it under >>> your-build-tree/Release+Asserts/bin/ or >>> your-build-tree/Debug+Asserts/bin/ or likewise, which depends on your >>> build >>> configuration. >>> >>> By the way, in the following ";RUN ..." directive >>> ; RUN: llc < %s -march=x86 | FileCheck %s >>> %s indicates current file under test. I guess the first %s stands for >>> the input for llc, while the second and last %s specifies >>> what to check using "; CHECK ..." directives since you write them down >>> on exactly CURRENT file. >>> >>> Hope it work for you. >>> Regards. >>> >>> 2012/8/20 Changcheng Wang <changcheng at multicorewareinc.com>: >>>> hi,all: >>>> i really want to how to write a regression test case,but i do not find >>>> out a entire document about it. >>>> By reading LLVM Testing Infrastructure Guide and FileCheck - Flexible >>>> pattern matching file verifier, i only have a idea,but it is not >>>> enough for me to write a test case.i need more detail about how to >>>> write a RUN:lines. >>>> >>>> thanks for you scan,wish your letter. >>>> >>>> best wishes, >>>> >>>> changcheng >>>> _______________________________________________ >>>> LLVM Developers mailing list >>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >>> >>> >>> >>> -- >>> 杨勇勇 (Yang Yongyong) > > > > -- > 杨勇勇 (Yang Yongyong)-- 杨勇勇 (Yang Yongyong)