llvm dev - Aug 2012 - [LLVMdev] where can I find out the documents of how to write a llvm regression test case?

Hi,
thanks for your letter,but i have read it several times,and now i am
confused still.because i do not know how the test work.
for a simple sample:

; RUN: llvm-as < %s | llvm-dis | FileCheck %s

@addr = external global i64

define i64 @add_unsigned(i64 %x, i64 %y) {
; CHECK: %z = add nuw i64 %x, %y
	%z = add nuw i64 %x, %y
	ret i64 %z
}

1.FileCheck verify the file that llvm-dis output and %s? if i just
want to verify if the file that llvm-dis output contain a string(such
as "abc“),what should i do?
2."; CHECK: %z = add nuw i64 %x, %y" and "%z = add nuw i64 %x,
%y" has
the same sentence, why?

so i really want to know more details about how to write a test,but i
can not find more documents out.

thanks

best wishes!

changcheng


On Mon, Aug 27, 2012 at 2:39 PM, Nadav Rotem <nrotem at apple.com>
wrote:
> The docs are here:  http://llvm.org/docs/TestingGuide.html#rtstructure . It
is a good idea to look at the existing regression tests in '/test'.
>
> On Aug 27, 2012, at 9:35 AM, Changcheng Wang <changcheng at
multicorewareinc.com> wrote:
>
>> hi,all:
>> I have try to write some regression tests of llvm,but it is hard for
>> me.because i have no knowledge about it and few documents about it.
>> if anyone can give me more documents about it.
>>
>> thanks
>>
>> best wishes
>>
>> changcheng
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

Hi Changcheng,

  Below is an example took from test/CodeGen/X86/add.ll:

define i32 @test1(i32 inreg %a) nounwind {
  %b = add i32 %a, 128
  ret i32 %b
; X32: subl     $-128, %eax
; X64: subl $-128,
}

  1. The first step, you write a function by using LLVM IR.

define i32 @test1(i32 inreg %a) nounwind {
  %b = add i32 %a, 128
  ret i32 %b
}

  This will be left to LLVM toolchain to generate binary code.   

  2. Then in the test case above, you add comment like,

; X32: subl     $-128, %eax
; X64: subl $-128,

  so that FileCheck can check the output of LLVM toolchain and what you
expect from it (; comment). O.K., let's go back to your questions,
> 1.FileCheck verify the file that llvm-dis output and %s? if i just
> want to verify if the file that llvm-dis output contain a string(such
> as "abc“),what should i do?
  %s means the *current* file, so you leave this file (*ll, human
readable LLVM IR) to llvm-as to generate *bc (bitcode, non-reabable
by human), use llvm-dis to get back *ll, then use Filecheck [1] to
compare its stdin (output of llvm-dis) with "; CHECK" line in %s
(i.e., this file). Usually you write what you expect in the test
case file, then let FileCheck to the work. The old way is using
`grep`, but it's not recommanded.
> 2."; CHECK: %z = add nuw i64 %x, %y" and "%z = add nuw i64
%x, %y" has
> the same sentence, why?
  Usually you add "; CHECK" in usual case before what you expect from
LLVM toolchain, or as the above example shows it adds "; X32:". This
is
one feature of FileCheck, you can specify what line you want to compare
with. Above test/CodeGen/X86/add.ll, you will see:

; RUN: llc < %s -mcpu=generic -march=x86 | FileCheck %s -check-prefix=X32

 which means FileCheck now will compare LLVM toolchain's result with lines
with"; X32:" prefix. But in your example, note below commond:

; RUN: llvm-as < %s | llvm-dis | FileCheck %s

It basically assemble LLVM IR first, pipe to llvm-dis to disassemble,
then use FileCheck to check if the content before llvm-as and after
llvm-dis is the same or not (should be the same), That's why you see

; CHECK: %z = add nuw i64 %x, %y
  %z = add nuw i64 %x, %y


Regards,
chenwj

[1] http://llvm.org/docs/TestingGuide.html#FileCheck

-- 
Wei-Ren Chen (陳韋任)
Computer Systems Lab, Institute of Information Science,
Academia Sinica, Taiwan (R.O.C.)
Tel:886-2-2788-3799 #1667
Homepage: http://people.cs.nctu.edu.tw/~chenwj

hi,chen:
thaks for your explaining,after reading it i have a few problems more.
1.in the sentenses of "X32: subl     $-128, %eax" and "; X64:
subl
$-128," ,i do not know what means that in detail,these sentences were
writen follow which language rules?
2.i want to write such a test:translate a *.ll(i.e:hello.ll) file to a
*.c file(i.e:hello.c) with llc,then verify if the .c file contian a
certain string(i.e:"abc:). so:
i want to change the old sample to achieve my aim,but i failed.my old
sample like this:

; RUN: llc -march=c < %s | FileCheck %s
; ModuleID = 'hello.c'
target datalayout
"e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"

@.str = private unnamed_addr constant [12 x i8] c"helloworld\0A\00",
align 1

define i32 @main() nounwind uwtable {
entry:
  %retval = alloca i32, align 4
  store i32 0, i32* %retval
  %call = call i32 (i8*, ...)* @printf(i8* getelementptr inbounds ([12
x i8]* @.str, i32 0, i32 0))
  ret i32 0
}

declare i32 @printf(i8*, ...)
;

;CHECK: main
;CHECK: printf

can you teach me how to do?
thanks !

changcheng

O n Mon, Aug 27, 2012 at 3:36 PM, 陳韋任 (Wei-Ren Chen)
<chenwj at iis.sinica.edu.tw> wrote:
> Hi Changcheng,
>
>   Below is an example took from test/CodeGen/X86/add.ll:
>
> define i32 @test1(i32 inreg %a) nounwind {
>   %b = add i32 %a, 128
>   ret i32 %b
> ; X32: subl     $-128, %eax
> ; X64: subl $-128,
> }
>
>   1. The first step, you write a function by using LLVM IR.
>
> define i32 @test1(i32 inreg %a) nounwind {
>   %b = add i32 %a, 128
>   ret i32 %b
> }
>
>   This will be left to LLVM toolchain to generate binary code.
>
>   2. Then in the test case above, you add comment like,
>
> ; X32: subl     $-128, %eax
> ; X64: subl $-128,
>
>   so that FileCheck can check the output of LLVM toolchain and what you
> expect from it (; comment). O.K., let's go back to your questions,
>
>> 1.FileCheck verify the file that llvm-dis output and %s? if i just
>> want to verify if the file that llvm-dis output contain a string(such
>> as "abc“),what should i do?
>
>   %s means the *current* file, so you leave this file (*ll, human
> readable LLVM IR) to llvm-as to generate *bc (bitcode, non-reabable
> by human), use llvm-dis to get back *ll, then use Filecheck [1] to
> compare its stdin (output of llvm-dis) with "; CHECK" line in %s
> (i.e., this file). Usually you write what you expect in the test
> case file, then let FileCheck to the work. The old way is using
> `grep`, but it's not recommanded.
>
>> 2."; CHECK: %z = add nuw i64 %x, %y" and "%z = add nuw
i64 %x, %y" has
>> the same sentence, why?
>
>   Usually you add "; CHECK" in usual case before what you expect
from
> LLVM toolchain, or as the above example shows it adds "; X32:".
This is
> one feature of FileCheck, you can specify what line you want to compare
> with. Above test/CodeGen/X86/add.ll, you will see:
>
> ; RUN: llc < %s -mcpu=generic -march=x86 | FileCheck %s
-check-prefix=X32
>
>  which means FileCheck now will compare LLVM toolchain's result with
lines
> with"; X32:" prefix. But in your example, note below commond:
>
> ; RUN: llvm-as < %s | llvm-dis | FileCheck %s
>
> It basically assemble LLVM IR first, pipe to llvm-dis to disassemble,
> then use FileCheck to check if the content before llvm-as and after
> llvm-dis is the same or not (should be the same), That's why you see
>
> ; CHECK: %z = add nuw i64 %x, %y
>   %z = add nuw i64 %x, %y
>
>
> Regards,
> chenwj
>
> [1] http://llvm.org/docs/TestingGuide.html#FileCheck
>
> --
> Wei-Ren Chen (陳韋任)
> Computer Systems Lab, Institute of Information Science,
> Academia Sinica, Taiwan (R.O.C.)
> Tel:886-2-2788-3799 #1667
> Homepage: http://people.cs.nctu.edu.tw/~chenwj