llvm dev - Dec 2008 - [LLVMdev] LLVM ARM Cross-Compiler Build

Mark Kromis wrote:
> On Dec 27, 2008, at 7:41 PM, Misha Brukman wrote:
>> 2008/12/27 Mark Kromis <greybird at mac.com>
>> Just a curiosity question, why push for gtest vs Boost Test or a 
>> different test suite?
>> I normally use Boost, and their test suite, so I'm more familiar
with
>> that. So I was wondering is one better then the other, or is it just 
>> that someone makes a patch for it?
I did a very minimal amount of research into unit testing frameworks 
before I started using gtest for my own work. Since then I've become 
quite familiar with gtest's features, which made it a natural choice for 
me. I presume that other testing frameworks will have their own 
defenders on this list, and I am certainly willing to listen to what 
they have to say.

The unit test comparison article mentioned earlier 
(http://gamesfromwithin.com/?p=29) is a good starting point for 
evaluating different unit test frameworks. Although gtest is too new to 
have been mentioned in the article directly, if you actually compare 
gtest's features against the various criteria used by the author of the 
article, it comes off quite well.

Note that the author of the article later went and created his own unit 
test framework (UnitTest++), based on his experience with all of the 
other frameworks he tested. This was the only framework that I seriously 
looked at other than gtest, and I found it to be a bit too minimal for 
what I needed.
>> I looked more into Boost.Test to see what's in it.  Boost.Test 
>> doesn't seem to be stand-alone -- I don't see a way to use
Boost.Test
>> without importing some other chunks of Boost that the testing library 
>> depends on.  While Boost is a fine set of libraries, I don't think
we
>> want to increase the LLVM distribution by sizeof(Boost) just to 
>> enable unittesting, nor do we want to spend the time on maintaining a 
>> subset of Boost that's "just enough" to build and use the
unittest
>> library, along a modified configure/build process that Boost wants to 
>> use (Boost.Build? Boost.Jam?).
Although I haven't actually tried Boost.Test, I kind of figured that 
this would be the case - that you pretty much have to drink the "Boost 
Kool-Aid" in order to use it.
> So are you planning on maintaining whatever test system, or just have 
> them as a pre-requisite. For example are you going to have the gtest 
> incorporated, or have them install it separately first? I was under 
> the impression that the user would have to install gtest first.
So the plan is to take a snapshot of gtest and check it in to the LLVM 
tree, rather than have it installed separately. I was able to integrate 
gtest into LLVM's build system fairly easily, as gtest is designed to be 
integrated into a foreign build system - basically I just ignored the 
makefile that comes with gtest, and wrote an LLVM-style makefile rule 
for it. There's a special source file in gtest which includes all other 
sources that is intended for just such a purpose. I did not need to 
modify the gtest sources in any way.

This means that keeping the gtest snapshot up to date will be trivial, 
since it will only require copying in the latest gtest snapshot and 
checking it in to LLVM - presuming that gtest remains backwards 
compatible, which I assume it will.

Licensing-wise, both LLVM and gtest are distributed under a fairly 
permissive BSD-style license. I don't know who would make the judgement 
call as to whether or not the licenses can co-exist. However, since 
neither license is "viral" in the sense of wanting to apply any sort
of
restriction on derived works or the "work as a whole", I see no
barrier
to shipping a combined product with different portions falling under 
different licenses. Thus, the unit tests themselves would still fall 
under the LLVM license, and linking the unit tests with gtest would not 
violate either license. Of course, IANAL.

 From a maintenance standpoint, we have already heard from several 
enthusiastic volunteers who are involved in both the development of LLVM 
and gtest. So I doubt there will be much problems on that score.

My personal goal is that one should be able to check out the head of 
LLVM on a generic Linux/OS-X system, with only the standard development 
environment (i.e. make/gcc/etc) and type "make unittest" and have the 
tests run.

In the longer term, I'd like to see LLVM have an automated build that 
runs the unit tests as part of the build. I noticed that gtest has an 
option to output test results in XML form, although I have not played 
with this personally, it might be useful in this regard.
>>
>> Boost also seems to want to use exceptions, and LLVM does not want 
>> to.  I'm not sure if there would be some difficulties in running a 
>> build where some libraries are compiled with no exceptions, some 
>> with, and the results are linked together.  At the best case, it 
>> would complicate our build system to be able to support different set 
>> of flags for building LLVM libraries vs. Boost.Test (and the rest of 
>> Boost that we import).
>
> http://www.boost.org/doc/libs/1_37_0/libs/utility/throw_exception.html
> #define BOOST_NO_EXCEPTIONS
>
>>
>> Sample usage of Boost.Test: 
>>
http://svn.boost.org/svn/boost/trunk/libs/test/example/unit_test_example_12.cpp
>>
>> Note the code at the end setting up the test suite -- this is 
>> boilerplate code that I think shouldn't be necessary to setup and
run
>> tests.
>>
>
>
>
http://svn.boost.org/svn/boost/trunk/libs/test/example/unit_test_example_01.cpp
>
> My test cases are not that in-depth, I'm much closer to sample 1. I 
> haven't found a reason to go that crazy yet.
>
>
>> Google Test, on the other hand, has no external dependencies, and is 
>> distributed as a dozen of .h/.cc files; supports Makefile, SCons, and 
>> Xcode; and doesn't use exceptions or RTTI.
>>
>
> Gtest is much more lightweight, no comparison there. I know that llvm 
> is not very good with exceptions, but should a test case system 
> support that?
>
>
>> Sample usage of GTest: 
>>
http://code.google.com/p/googletest/source/browse/trunk/samples/sample5_unittest.cc
>>
>> GTest-specific LOC besides the #include statement: 0.
>
> I think it links to a library as well.
>
>> Note that I'm not counting main() for either Boost or GTest,
because
>> both provide a standard main() for use with almost all test files.
>>
>> Misha
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
> Also for a note of reference, your links to the examples are the most 
> advanced samples. So boost can do more, thus has more weight/bloat 
> behind it.
>
> Were the other test kits looked at? Is gtest the best solution for the 
> project.
>
> Is this something your planning as putting in the tree, thus require 
> pulling in changes from google (license allowing), or does user need 
> to have the libraries/headers pre-installed?
>
> My question was not to cause a battle, but I wanted to be sure we were 
> using the right test kit, and not just picking one just because.  For 
> example gtest is very light weight test kit, that can do the job, but 
> will the tests outgrow what the test kit can do, and cause a 
> conversion to a more advanced one later?
>
> Regards,
> Mark Kromis
> ------------------------------------------------------------------------
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

Hi,

 This is a simple question about building ARM cross-compiler.
 What is the building procedure using LLVM 2.4 and GCC front-end 4.2?

 I used these commands for LLVM, and it is okay.

 $ ../configure --prefix=/usr/local -target=arm
 $ make ENABLE_OPTIMIZED=0
 $ make ENABLE_OPTIMIZED=0 install

 Then these commands were used for GCC front-end but this got couple of
errors.

 $ ../configure --prefix=/usr/local --enable-checking
--enable-llvm=../../llvm/objdir/ --disable-bootstrap --disable-multilib
--program-prefix=llvm- --target=arm-linux-gnu --enable-languages=c,c++,objc
 $ make; make install
 
 Compiler errors are caused due to absence of definition of macros:
 MACHO_DYNAMIC_NO_PIC_P, TARGET_64BIT, and so on.

 When these macros are manually added, it caused another error, like below:

 ../objdir/./gcc/xgcc -B../objdir/./gcc/ -B/usr/local/arm-linux-gnu/bin/
-B/usr/local/arm-linux-gnu/lib/ -isystem /usr/local/arm-linux-gnu/include
-isystem /usr/local/arm-linux-gnu/sys-include -O2 -O2 -g -O2 -DIN_GCC
-DCROSS_DIRECTORY_STRUCTURE -W -Wall -Wwrite-strings -Wstrict-prototypes
-Wmissing-prototypes -Wold-style-definition -isystem ./include  -I. -I.
-I../../gcc -I../../gcc/. -I../../gcc/../include
-I../../gcc/../libcpp/include  -I../../gcc/../libdecnumber -I../libdecnumber
-I../objdir//include -../../llvm-2.4/include  -g0 -finhibit-size-directive
-fno-inline-functions -fno-exceptions -fno-zero-initialized-in-bss
-fno-toplevel-reorder -Dinhibit_libc  \
          -c ../../gcc/crtstuff.c -DCRT_BEGIN \
          -o crtbegin.o
../objdir/./gcc/as: line 2: exec: -o: invalid option
exec: usage: exec [-cl] [-a name] file [redirection ...]
make[2]: *** [crtbegin.o] Error 1
make[2]: Leaving directory `../objdir/gcc'

 As a native GCC, 4.3.0, 3.4.2, and 3.4.6 were used but all have same error.
 A similar command except for --target=arm option works correctly.

 In addition to that, does any developer succeed in compiling a micro-kernel
(e.g., L4 or Xen/ARM) using LLVM ARM port?

 Thanks in advance.

 Best regards,
 Keun Soo

On Dec 27, 2008, at 8:54 PM, Talin wrote:
> Although I haven't actually tried Boost.Test, I kind of figured that
> this would be the case - that you pretty much have to drink the "Boost
> Kool-Aid" in order to use it.
I agree, boost.test seems like a non-starter from many reasons.
>> So are you planning on maintaining whatever test system, or just have
>> them as a pre-requisite. For example are you going to have the gtest
>> incorporated, or have them install it separately first? I was under
>> the impression that the user would have to install gtest first.
> So the plan is to take a snapshot of gtest and check it in to the LLVM
> tree, rather than have it installed separately.
Yes, we'd want to either do this, or have the configure machinery  
detect an installed copy and only enable unit testing if present.  If  
the goal is to make everyone run unit tests, embedding a copy would be  
the best way to go.
> This means that keeping the gtest snapshot up to date will be trivial,
> since it will only require copying in the latest gtest snapshot and
> checking it in to LLVM - presuming that gtest remains backwards
> compatible, which I assume it will.
This is only an issue if the latest and greatest versions of gtest add  
something we need.  My understanding of unit testing is fairly  
limited, but I don't forsee major new innovations regularly happening  
that we'll need to bring in on a frequent basis.  Doing a periodic  
synch (once every year or two) will probably be fine.
>
> Licensing-wise, both LLVM and gtest are distributed under a fairly
> permissive BSD-style license. I don't know who would make the  
> judgement
> call as to whether or not the licenses can co-exist. However, since
> neither license is "viral" in the sense of wanting to apply any
sort
> of
> restriction on derived works or the "work as a whole", I see no  
> barrier
> to shipping a combined product with different portions falling under
> different licenses. Thus, the unit tests themselves would still fall
> under the LLVM license, and linking the unit tests with gtest would  
> not
> violate either license. Of course, IANAL.
The licenses coexist.  Please just add an entry to the llvm/ 
LICENSES.txt file.
> My personal goal is that one should be able to check out the head of
> LLVM on a generic Linux/OS-X system, with only the standard  
> development
> environment (i.e. make/gcc/etc) and type "make unittest" and have
the
> tests run.
Yep.  BTW, do these tests run in parallel with make -jN?  If not,  
please make them :)
> In the longer term, I'd like to see LLVM have an automated build that
> runs the unit tests as part of the build. I noticed that gtest has an
> option to output test results in XML form, although I have not played
> with this personally, it might be useful in this regard.
The only thing I'm concerned about here is extending build time.  For  
example, if we have a lot of unit tests for libsupport (e.g.  
densemap), I wouldn't want those tests to be run unless libsupport  
changes.  It should just be a "small matter of makefile hackery" to  
get this going I suppose.

Is there a final proposed version of the patch with the changes Misha  
asked for?  If so, I'll review and commit it when I have time.

-Chris

I'm working on an update to the patch. The only thing holding me up is
trying to come to a final decision as to where all the various pieces should
live. Specifically, the Google Test library, and the actual unit tests
themselves.
On Mon, Dec 29, 2008 at 9:17 AM, Chris Lattner <clattner at apple.com>
wrote:
> On Dec 27, 2008, at 8:54 PM, Talin wrote:
> > Although I haven't actually tried Boost.Test, I kind of figured
that
> > this would be the case - that you pretty much have to drink the
"Boost
> > Kool-Aid" in order to use it.
>
> I agree, boost.test seems like a non-starter from many reasons.
>
> >> So are you planning on maintaining whatever test system, or just
have
> >> them as a pre-requisite. For example are you going to have the
gtest
> >> incorporated, or have them install it separately first? I was
under
> >> the impression that the user would have to install gtest first.
> > So the plan is to take a snapshot of gtest and check it in to the LLVM
> > tree, rather than have it installed separately.
>
> Yes, we'd want to either do this, or have the configure machinery
> detect an installed copy and only enable unit testing if present.  If
> the goal is to make everyone run unit tests, embedding a copy would be
> the best way to go.
>
> > This means that keeping the gtest snapshot up to date will be trivial,
> > since it will only require copying in the latest gtest snapshot and
> > checking it in to LLVM - presuming that gtest remains backwards
> > compatible, which I assume it will.
>
> This is only an issue if the latest and greatest versions of gtest add
> something we need.  My understanding of unit testing is fairly
> limited, but I don't forsee major new innovations regularly happening
> that we'll need to bring in on a frequent basis.  Doing a periodic
> synch (once every year or two) will probably be fine.
>
> >
> > Licensing-wise, both LLVM and gtest are distributed under a fairly
> > permissive BSD-style license. I don't know who would make the
> > judgement
> > call as to whether or not the licenses can co-exist. However, since
> > neither license is "viral" in the sense of wanting to apply
any sort
> > of
> > restriction on derived works or the "work as a whole", I see
no
> > barrier
> > to shipping a combined product with different portions falling under
> > different licenses. Thus, the unit tests themselves would still fall
> > under the LLVM license, and linking the unit tests with gtest would
> > not
> > violate either license. Of course, IANAL.
>
> The licenses coexist.  Please just add an entry to the llvm/
> LICENSES.txt file.
>
> > My personal goal is that one should be able to check out the head of
> > LLVM on a generic Linux/OS-X system, with only the standard
> > development
> > environment (i.e. make/gcc/etc) and type "make unittest" and
have the
> > tests run.
>
> Yep.  BTW, do these tests run in parallel with make -jN?  If not,
> please make them :)
>
> > In the longer term, I'd like to see LLVM have an automated build
that
> > runs the unit tests as part of the build. I noticed that gtest has an
> > option to output test results in XML form, although I have not played
> > with this personally, it might be useful in this regard.
>
> The only thing I'm concerned about here is extending build time.  For
> example, if we have a lot of unit tests for libsupport (e.g.
> densemap), I wouldn't want those tests to be run unless libsupport
> changes.  It should just be a "small matter of makefile hackery"
to
> get this going I suppose.
>
> Is there a final proposed version of the patch with the changes Misha
> asked for?  If so, I'll review and commit it when I have time.
>
> -Chris
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>


-- 
-- Talin
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20081229/e8dafd6a/attachment.html>

2008/12/28 Keun Soo Yim <yim6 at illinois.edu>
>  This is a simple question about building ARM cross-compiler.
>  What is the building procedure using LLVM 2.4 and GCC front-end 4.2?
>
>  I used these commands for LLVM, and it is okay.
>
>  $ ../configure --prefix=/usr/local -target=arm
>  $ make ENABLE_OPTIMIZED=0
>  $ make ENABLE_OPTIMIZED=0 install
>
>  Then these commands were used for GCC front-end but this got couple of
> errors.
>
>  $ ../configure --prefix=/usr/local --enable-checking
> --enable-llvm=../../llvm/objdir/ --disable-bootstrap --disable-multilib
> --program-prefix=llvm- --target=arm-linux-gnu --enable-languages=c,c++,objc
>  $ make; make install
>
>  Compiler errors are caused due to absence of definition of macros:
>  MACHO_DYNAMIC_NO_PIC_P, TARGET_64BIT, and so on.
>
I've reproduced the same error on my x86/Linux system (the first macro is
undefined).  I suspect if I go ahead and also add it, I'll continue to the
next error you found.  One might guess that since this cross-tool build is
so broken, that it's likely not used by the folks targeting the ARM backend.

The folks developing on ARM instead might be using QEMU to emulate ARM on
x86, and run it as a regular (not cross) compiler in that environment.  In
fact, looking at the nightly tester runs, I've noticed one of the ARM
nightly testers seems to run in QEMU:

Nickname:shark-qemu-armv5tel
uname:Linux shark-armel 2.6.24-rc7 #1 Wed Jan 16 02:28:08 CET 2008 armv5tel
GNU/Linux
http://llvm.org/nightlytest/machine.php?machine=335

However, some other nightly testers may be running cross-compilers (but are
probably still running the tests via QEMU or similar emulator):

Nickname:lauro-armv6-N800
uname:Linux laurov-laptop 2.6.20-16-generic #2 SMP Thu Jun 7 20:19:32 UTC
2007 i686 GNU/Linux
http://llvm.org/nightlytest/machine.php?machine=172

Nickname:lauro-armv5te-softfloat
uname:Linux laurov-desktop 2.6.17-11-generic #2 SMP Tue Mar 13 23:32:38 UTC
2007 i686 GNU/Linux
http://llvm.org/nightlytest/machine.php?machine=121

Perhaps the owner(s) of these nightly testers would know how to do what
you're trying to accomplish.

Misha
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20081230/cf4b7af3/attachment.html>