llvm dev - Aug 2012 - [LLVMdev] where can I find out the documents of how to write a llvm regression test case?

hi,chen:
thaks for your explaining,after reading it i have a few problems more.
1.in the sentenses of "X32: subl     $-128, %eax" and "; X64:
subl
$-128," ,i do not know what means that in detail,these sentences were
writen follow which language rules?
2.i want to write such a test:translate a *.ll(i.e:hello.ll) file to a
*.c file(i.e:hello.c) with llc,then verify if the .c file contian a
certain string(i.e:"abc:). so:
i want to change the old sample to achieve my aim,but i failed.my old
sample like this:

; RUN: llc -march=c < %s | FileCheck %s
; ModuleID = 'hello.c'
target datalayout
"e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"

@.str = private unnamed_addr constant [12 x i8] c"helloworld\0A\00",
align 1

define i32 @main() nounwind uwtable {
entry:
  %retval = alloca i32, align 4
  store i32 0, i32* %retval
  %call = call i32 (i8*, ...)* @printf(i8* getelementptr inbounds ([12
x i8]* @.str, i32 0, i32 0))
  ret i32 0
}

declare i32 @printf(i8*, ...)
;

;CHECK: main
;CHECK: printf

can you teach me how to do?
thanks !

changcheng

O n Mon, Aug 27, 2012 at 3:36 PM, 陳韋任 (Wei-Ren Chen)
<chenwj at iis.sinica.edu.tw> wrote:
> Hi Changcheng,
>
>   Below is an example took from test/CodeGen/X86/add.ll:
>
> define i32 @test1(i32 inreg %a) nounwind {
>   %b = add i32 %a, 128
>   ret i32 %b
> ; X32: subl     $-128, %eax
> ; X64: subl $-128,
> }
>
>   1. The first step, you write a function by using LLVM IR.
>
> define i32 @test1(i32 inreg %a) nounwind {
>   %b = add i32 %a, 128
>   ret i32 %b
> }
>
>   This will be left to LLVM toolchain to generate binary code.
>
>   2. Then in the test case above, you add comment like,
>
> ; X32: subl     $-128, %eax
> ; X64: subl $-128,
>
>   so that FileCheck can check the output of LLVM toolchain and what you
> expect from it (; comment). O.K., let's go back to your questions,
>
>> 1.FileCheck verify the file that llvm-dis output and %s? if i just
>> want to verify if the file that llvm-dis output contain a string(such
>> as "abc“),what should i do?
>
>   %s means the *current* file, so you leave this file (*ll, human
> readable LLVM IR) to llvm-as to generate *bc (bitcode, non-reabable
> by human), use llvm-dis to get back *ll, then use Filecheck [1] to
> compare its stdin (output of llvm-dis) with "; CHECK" line in %s
> (i.e., this file). Usually you write what you expect in the test
> case file, then let FileCheck to the work. The old way is using
> `grep`, but it's not recommanded.
>
>> 2."; CHECK: %z = add nuw i64 %x, %y" and "%z = add nuw
i64 %x, %y" has
>> the same sentence, why?
>
>   Usually you add "; CHECK" in usual case before what you expect
from
> LLVM toolchain, or as the above example shows it adds "; X32:".
This is
> one feature of FileCheck, you can specify what line you want to compare
> with. Above test/CodeGen/X86/add.ll, you will see:
>
> ; RUN: llc < %s -mcpu=generic -march=x86 | FileCheck %s
-check-prefix=X32
>
>  which means FileCheck now will compare LLVM toolchain's result with
lines
> with"; X32:" prefix. But in your example, note below commond:
>
> ; RUN: llvm-as < %s | llvm-dis | FileCheck %s
>
> It basically assemble LLVM IR first, pipe to llvm-dis to disassemble,
> then use FileCheck to check if the content before llvm-as and after
> llvm-dis is the same or not (should be the same), That's why you see
>
> ; CHECK: %z = add nuw i64 %x, %y
>   %z = add nuw i64 %x, %y
>
>
> Regards,
> chenwj
>
> [1] http://llvm.org/docs/TestingGuide.html#FileCheck
>
> --
> Wei-Ren Chen (陳韋任)
> Computer Systems Lab, Institute of Information Science,
> Academia Sinica, Taiwan (R.O.C.)
> Tel:886-2-2788-3799 #1667
> Homepage: http://people.cs.nctu.edu.tw/~chenwj

> thaks for your explaining,after reading it i have a few problems more.
> 1.in the sentenses of "X32: subl     $-128, %eax" and ";
X64: subl
> $-128," ,i do not know what means that in detail,these sentences were
> writen follow which language rules?
  Accroding to what backend developers expect from the backend he wrote
and architecture manual.
> 2.i want to write such a test:translate a *.ll(i.e:hello.ll) file to a
> *.c file(i.e:hello.c) with llc,then verify if the .c file contian a
> certain string(i.e:"abc:). so:
> i want to change the old sample to achieve my aim,but i failed.my old
> sample like this:
> 
> ; RUN: llc -march=c < %s | FileCheck %s
> ; ModuleID = 'hello.c'
> target datalayout >
"e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
> target triple = "x86_64-unknown-linux-gnu"
> 
> @.str = private unnamed_addr constant [12 x i8]
c"helloworld\0A\00", align 1
> 
> define i32 @main() nounwind uwtable {
> entry:
>   %retval = alloca i32, align 4
>   store i32 0, i32* %retval
>   %call = call i32 (i8*, ...)* @printf(i8* getelementptr inbounds ([12
> x i8]* @.str, i32 0, i32 0))
>   ret i32 0
> }
> 
> declare i32 @printf(i8*, ...)
> ;
> 
> ;CHECK: main
> ;CHECK: printf
  The first thing you should know is C backend had been removed from LLVM
since LLVM 3.1 [1], I don't know what you want is doable... Maybe you can
send a new mail or just reply this mail with a different subject like

  Subject: Writing test case for LLVM C backend

That will make your intension more explicit. If you're using a *work*
LLVM C backend, have you tried to run `llc -march=c hello.ll` manually
and see what the output is? Are you sure the C backend output is the
same under any enviroment? I mean maybe when you run this test case on
different OS/ISA, then result could be different. And have you tried
the following?

  ;CHECK: int main()
  define i32 @main() nounwind uwtable { 

    ... snip ...

  }

Regards,
chenwj

[1] http://llvm.org/releases/3.1/docs/ReleaseNotes.html

-- 
Wei-Ren Chen (陳韋任)
Computer Systems Lab, Institute of Information Science,
Academia Sinica, Taiwan (R.O.C.)
Tel:886-2-2788-3799 #1667
Homepage: http://people.cs.nctu.edu.tw/~chenwj

> 2.i want to write such a test:translate a *.ll(i.e:hello.ll) file to a
> *.c file(i.e:hello.c) with llc,then verify if the .c file contian a
> certain string(i.e:"abc:). so:
  It seems you want to test LLVM C backend, go looking for
llvm-3.0.src/test/CodeGen/CBackend/* (C backend had been
removed from LLVM 3.1).

HTH,
chenwj

-- 
Wei-Ren Chen (陳韋任)
Computer Systems Lab, Institute of Information Science,
Academia Sinica, Taiwan (R.O.C.)
Tel:886-2-2788-3799 #1667
Homepage: http://people.cs.nctu.edu.tw/~chenwj

i am very surprised that you have known what i want to do,you are capable!
thank you very much.

yours

changcheng


On Mon, Aug 27, 2012 at 5:37 PM, 陳韋任 (Wei-Ren Chen)
<chenwj at iis.sinica.edu.tw> wrote:
>> 2.i want to write such a test:translate a *.ll(i.e:hello.ll) file to a
>> *.c file(i.e:hello.c) with llc,then verify if the .c file contian a
>> certain string(i.e:"abc:). so:
>
>   It seems you want to test LLVM C backend, go looking for
> llvm-3.0.src/test/CodeGen/CBackend/* (C backend had been
> removed from LLVM 3.1).
>
> HTH,
> chenwj
>
> --
> Wei-Ren Chen (陳韋任)
> Computer Systems Lab, Institute of Information Science,
> Academia Sinica, Taiwan (R.O.C.)
> Tel:886-2-2788-3799 #1667
> Homepage: http://people.cs.nctu.edu.tw/~chenwj

hi,weiren:
i have reading the samples in  llvm-3.0.src/test/CodeGen/CBackend/.
and i have two questions,
1. int the file of 2008-02-01-UnalignedLoadStore.ll,have sentences as below:
; RUN: llc < %s -march=c | \
; RUN:          grep {struct __attribute__ ((packed, aligned(} | count 4

if " grep {struct __attribute__ ((packed, aligned(}" means to verify
the whole file(llc output) to find out the string of "struct
__attribute__ ((packed, aligned("?
what is the mean of "count 4"?

2.i write a test case with FileCheck like below,but fail:
 ; RUN: llc -march=c < %s | FileCheck %s

; ModuleID = 'hello.c'
target datalayout
"e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"

@.str = private unnamed_addr constant [12 x i8] c"helloworld\0A\00",
align 1

define i32 @main() nounwind uwtable {
entry:
  %retval = alloca i32, align 4
  store i32 0, i32* %retval
  %call = call i32 (i8*, ...)* @printf(i8* getelementptr inbounds ([12
x i8]* @.str, i32 0, i32 0))
  ret i32 0
}

declare i32 @printf(i8*, ...)
;

;CHECK: main
;CHECK: printf
;CHECK: {define}

i means to check the string of "define" in the file(llc output).i do
not know where is wrong,can you help me?

thanks,

changcheng


On Mon, Aug 27, 2012 at 5:37 PM, 陳韋任 (Wei-Ren Chen)
<chenwj at iis.sinica.edu.tw> wrote:
>> 2.i want to write such a test:translate a *.ll(i.e:hello.ll) file to a
>> *.c file(i.e:hello.c) with llc,then verify if the .c file contian a
>> certain string(i.e:"abc:). so:
>
>   It seems you want to test LLVM C backend, go looking for
> llvm-3.0.src/test/CodeGen/CBackend/* (C backend had been
> removed from LLVM 3.1).
>
> HTH,
> chenwj
>
> --
> Wei-Ren Chen (陳韋任)
> Computer Systems Lab, Institute of Information Science,
> Academia Sinica, Taiwan (R.O.C.)
> Tel:886-2-2788-3799 #1667
> Homepage: http://people.cs.nctu.edu.tw/~chenwj