llvm dev - Jul 2009 - [LLVMdev] Unexpected failures in the DejaGNU test collection

Since it came up, what is blocking us from putting these tests in the
llvm-gcc test suite instead of in the LLVM test suite?

 - Daniel

On Tue, Jul 14, 2009 at 4:01 AM, Jakob Stoklund Olesen<stoklund at 2pi.dk>
wrote:
>
> On 14/07/2009, at 12.35, Harel Cain wrote:
>> When using "make check" with the DejaGNU test collection, I
encounter
>> two unexpected failures (they seem to be closely related).
>> My question: are they well known, and if so what's the problem and
how
>> can I fix it?
>
>> FAIL: /var/data/common/trunk/llvm/test/FrontendC/2008-05-19-
>> AlwaysInline.c
>> FAIL: /var/data/common/trunk/llvm/test/FrontendC/always-inline.c
>
> Hello Harel,
>
> Failures in the FrontendC test suite usually means that your llvm-gcc
> is older than your LLVM tree.
>
> Try compiling llvm-gcc from top-of-tree and see if the failures remain.
>
> Regards,
> /jakob
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

On Tue, Jul 14, 2009 at 8:49 AM, Daniel Dunbar<daniel at zuster.org>
wrote:
> Since it came up, what is blocking us from putting these tests in the
> llvm-gcc test suite instead of in the LLVM test suite?
Usually we want to check these tests regularly against llvm backend
changes. The nightly tester does not svn update and rebuild llvm-gcc
hence these tests are in LLVM Test Suite. BTW, llvm-gcc test suite
already has llvm.objc and llvm-obj-c++ folders for llvm related tests.

-
Devang

On Jul 14, 2009, at 10:03 AM, Devang Patel wrote:
> On Tue, Jul 14, 2009 at 8:49 AM, Daniel Dunbar<daniel at zuster.org>
> wrote:
>> Since it came up, what is blocking us from putting these tests in the
>> llvm-gcc test suite instead of in the LLVM test suite?
>
> Usually we want to check these tests regularly against llvm backend
> changes. The nightly tester does not svn update and rebuild llvm-gcc
> hence these tests are in LLVM Test Suite. BTW, llvm-gcc test suite
> already has llvm.objc and llvm-obj-c++ folders for llvm related tests.
>
Do the nightly testers even run the llvm-gcc test suite or keep track  
of those results? I'm pretty sure they don't.

It is also not our release criteria. So something to keep in mind.

-Tanya
> -
> Devang
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20090714/1a708868/attachment.html>

Hi Daniel,
> Since it came up, what is blocking us from putting these tests in the
> llvm-gcc test suite instead of in the LLVM test suite?
they are useful for testing the llvm gcc plugin.  This is llvm-gcc
implemented as a gcc plugin.  As such, it is not possible to make
modifications to the underlying gcc tree, such as adding testcases,
except by pushing changes upstream to mainline gcc.  I doubt they will
except llvm-gcc testcases, but you never know :)

Ciao,

Duncan.

On Tue, Jul 14, 2009 at 10:03 AM, Devang Patel<devang.patel at gmail.com>
wrote:
> On Tue, Jul 14, 2009 at 8:49 AM, Daniel Dunbar<daniel at zuster.org>
wrote:
>> Since it came up, what is blocking us from putting these tests in the
>> llvm-gcc test suite instead of in the LLVM test suite?
>
> Usually we want to check these tests regularly against llvm backend
> changes.
Thats fine, but meaningless. If the backend changes and llvm-gcc isn't
built, it accomplishes nothing. There is a fundamental flaw in
checking in the tests for tool FOO into the repository for tool BAR.
> The nightly tester does not svn update and rebuild llvm-gcc
> hence these tests are in LLVM Test Suite.
This is a solvable problem.
> BTW, llvm-gcc test suite already has llvm.objc and llvm-obj-c++ folders for
> llvm related tests.
Ok.

 - Daniel
> -
> Devang
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

On Wed, Jul 15, 2009 at 2:17 AM, Duncan Sands<baldrick at free.fr>
wrote:
> Hi Daniel,
>
>> Since it came up, what is blocking us from putting these tests in the
>> llvm-gcc test suite instead of in the LLVM test suite?
>
> they are useful for testing the llvm gcc plugin.  This is llvm-gcc
> implemented as a gcc plugin.  As such, it is not possible to make
> modifications to the underlying gcc tree, such as adding testcases,
> except by pushing changes upstream to mainline gcc.  I doubt they will
> except llvm-gcc testcases, but you never know :)
Tests for the llvm gcc plugin should be in the llvm gcc plugin repository.

I think it is worth solving this problem, because it currently makes
the public buildbot results less than useful, it frequently breaks on
Frontend changes. I also personally find this pretty annoying that I
am forced to update llvm-gcc more frequently than I would otherwise
need to, if I want clean local test results. I don't think most LLVM
developers would appreciate clang tests being in the LLVM repo,
either.

 - Daniel
>
> Ciao,
>
> Duncan.
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>