llvm dev - Nov 2012 - [LLVMdev] Need help reading the LLVM Buildbot results

>From r167602 and onwards I get a fail in 'make check-all' for
llvm+clang, built with gcc-4.7.1 (but not with clang-3.1) on Linux x86_64:
Failing Tests (1):
    Clang :: CodeGenCXX/mangle-ms-templates.cpp

clang: /dev/shm/uabpath/master/include/llvm/Support/Casting.h:208: typename
llvm::cast_retty<To, From>::ret_type llvm::cast(const Y&) [with X =
clang::TemplateSpecializationType; Y = const clang::Type*; typename
llvm::cast_retty<To, From>::ret_type = const
clang::TemplateSpecializationType*]: Assertion `isa<X>(Val) &&
"cast<Ty>() argument of incompatible type!"' failed.

This made me take a look at the LLVM Buildbot pages at
http://lab.llvm.org:8011/. However, I find it difficult to find useful
information matching this problem.

* Can I look up results for r167602 (or closely afterwards)? (Can I compare it
with r167601 or less?)
* How can I see if some buildbot is using gcc-4.7.1 (or later)?
* There seems to be quite a few failed builds in the buildbots. Which buildbot
configurations should I expect to be maintained to produce mostly non-failed
builds?
* Is there some general introduction document for reading the LLVM Buildbot
results?

Regards,
Patrik Hägglund


-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20121110/ba3a5603/attachment.html>

On Sat, Nov 10, 2012 at 3:21 AM, Patrik Hägglund H
<patrik.h.hagglund at ericsson.com> wrote:
> From r167602 and onwards I get a fail in 'make check-all' for
llvm+clang,
> built with gcc-4.7.1 (but not with clang-3.1) on Linux x86_64:
>
> Failing Tests (1):
>     Clang :: CodeGenCXX/mangle-ms-templates.cpp
>
> clang: /dev/shm/uabpath/master/include/llvm/Support/Casting.h:208: typename
> llvm::cast_retty<To, From>::ret_type llvm::cast(const Y&) [with X
> clang::TemplateSpecializationType; Y = const clang::Type*; typename
> llvm::cast_retty<To, From>::ret_type = const
> clang::TemplateSpecializationType*]: Assertion `isa<X>(Val)
&& "cast<Ty>()
> argument of incompatible type!"' failed.
If you take the full command line out of the build log & run it
separately you should be able to reproduce this assert. Depending on
the complexity of the test, you may be able to simply remove the
"-cc1" that's probably in the command line & watch it fail but
this
time you'll get a crash report from Clang including the path to a .sh
file containing the command line and a .cpp file containing the
preprocessed source code necessary to reproduce the problem. If you
can file that in a bug we should be able to reproduce & fix it.
> This made me take a look at the LLVM Buildbot pages at
> http://lab.llvm.org:8011/. However, I find it difficult to find useful
> information matching this problem.
Don't worry, you're not the only one. This area could use some
improvement (& there's some amount of work going on in that area)
> * Can I look up results for r167602 (or closely afterwards)? (Can I compare
> it with r167601 or less?)
I don't believe so - I think only a certain sliding window of logs are
kept (& I don't know of an easy way to view a particular revision
across the bots either)
> * How can I see if some buildbot is using gcc-4.7.1 (or later)?
I don't believe so - some bots indicate something of their
purpose/role in the name of the builder, but it's not a well specified
attribute. I suppose we could add something to the builder profile
that causes us to run a version check on the host compiler (but that
would only show up in the build logs at best, not in the builder
description). Anything else I think would be manually maintained
(which might still be worthwhile)
> * There seems to be quite a few failed builds in the buildbots. Which
> buildbot configurations should I expect to be maintained to produce mostly
> non-failed builds?
Unclear - generally you can look at the finite history shown on the
buildbot console page & if a bot's been red for that entire time
it's
probably about worth ignoring (until someone takes the time to clean
it up). We should move the dead buildbots aside until someone revives
them, rather than leaving them there adding noise to the page.
> * Is there some general introduction document for reading the LLVM Buildbot
> results?
Not that I know of.

Thanks David,
> file that in a bug
I have filed TR 14321. http://llvm.org/bugs/show_bug.cgi?id=14321

(Just removing -cc1 altered the behavior.)
> Don't worry, you're not the only one.
Good. :-)

/Patrik Hägglund

-----Original Message-----
From: David Blaikie [mailto:dblaikie at gmail.com] 
Sent: den 12 november 2012 05:19
To: Patrik Hägglund H
Cc: llvmdev at cs.uiuc.edu; nicolasweber at gmx.de
Subject: Re: [LLVMdev] Need help reading the LLVM Buildbot results

On Sat, Nov 10, 2012 at 3:21 AM, Patrik Hägglund H
<patrik.h.hagglund at ericsson.com> wrote:
> From r167602 and onwards I get a fail in 'make check-all' for
llvm+clang,
> built with gcc-4.7.1 (but not with clang-3.1) on Linux x86_64:
>
> Failing Tests (1):
>     Clang :: CodeGenCXX/mangle-ms-templates.cpp
>
> clang: /dev/shm/uabpath/master/include/llvm/Support/Casting.h:208: typename
> llvm::cast_retty<To, From>::ret_type llvm::cast(const Y&) [with X
> clang::TemplateSpecializationType; Y = const clang::Type*; typename
> llvm::cast_retty<To, From>::ret_type = const
> clang::TemplateSpecializationType*]: Assertion `isa<X>(Val)
&& "cast<Ty>()
> argument of incompatible type!"' failed.
If you take the full command line out of the build log & run it
separately you should be able to reproduce this assert. Depending on
the complexity of the test, you may be able to simply remove the
"-cc1" that's probably in the command line & watch it fail but
this
time you'll get a crash report from Clang including the path to a .sh
file containing the command line and a .cpp file containing the
preprocessed source code necessary to reproduce the problem. If you
can file that in a bug we should be able to reproduce & fix it.
> This made me take a look at the LLVM Buildbot pages at
> http://lab.llvm.org:8011/. However, I find it difficult to find useful
> information matching this problem.
Don't worry, you're not the only one. This area could use some
improvement (& there's some amount of work going on in that area)
> * Can I look up results for r167602 (or closely afterwards)? (Can I compare
> it with r167601 or less?)
I don't believe so - I think only a certain sliding window of logs are
kept (& I don't know of an easy way to view a particular revision
across the bots either)
> * How can I see if some buildbot is using gcc-4.7.1 (or later)?
I don't believe so - some bots indicate something of their
purpose/role in the name of the builder, but it's not a well specified
attribute. I suppose we could add something to the builder profile
that causes us to run a version check on the host compiler (but that
would only show up in the build logs at best, not in the builder
description). Anything else I think would be manually maintained
(which might still be worthwhile)
> * There seems to be quite a few failed builds in the buildbots. Which
> buildbot configurations should I expect to be maintained to produce mostly
> non-failed builds?
Unclear - generally you can look at the finite history shown on the
buildbot console page & if a bot's been red for that entire time
it's
probably about worth ignoring (until someone takes the time to clean
it up). We should move the dead buildbots aside until someone revives
them, rather than leaving them there adding noise to the page.
> * Is there some general introduction document for reading the LLVM Buildbot
> results?
Not that I know of.

On 12 November 2012 04:18, David Blaikie <dblaikie at gmail.com>
wrote:
>> * Can I look up results for r167602 (or closely afterwards)? (Can I
compare
>> it with r167601 or less?)
>
> I don't believe so - I think only a certain sliding window of logs are
> kept (& I don't know of an easy way to view a particular revision
> across the bots either)
That'd be a very nice feature!

--renato

On Mon, Nov 12, 2012 at 7:03 AM, David Tweed <david.tweed at arm.com>
wrote:
>> I don't believe so - I think only a certain sliding window of logs
are
>> kept (& I don't know of an easy way to view a particular
revision
>> across the bots either)
>
> For me a bigger issue has been inability to see a long list of results for
a
> particular builder (say, arm-cortex-a9-blah). Unfortunately while I can see
> "the last 20 builds over all targets" I can only see "the
last five builds
> for a particular builder". What's frustrating is that I'm
pretty sure the
> buildbot is keeping the info, I just can't see how to get it.
When you're on a builder's page (eg:
http://lab.llvm.org:8011/builders/clang-x86_64-debian ) you can "view
in waterfall" (
http://lab.llvm.org:8011/waterfall?show=clang-x86_64-debian ) which
gives you more history (even on the first page it's got about 14
builds & it has multiple pages to go back further) but it's the high
detail waterfall view, not a simple red/green build status summary of
each build.

Having more history in the "recent builds" would be nice (or somewhere
else where that could be easily visualized)

On Mon, Nov 12, 2012 at 8:19 AM, David Blaikie <dblaikie at gmail.com>
wrote:
> On Mon, Nov 12, 2012 at 7:03 AM, David Tweed <david.tweed at arm.com>
wrote:
>>> I don't believe so - I think only a certain sliding window of
logs are
>>> kept (& I don't know of an easy way to view a particular
revision
>>> across the bots either)
>>
>> For me a bigger issue has been inability to see a long list of results
for a
>> particular builder (say, arm-cortex-a9-blah). Unfortunately while I can
see
>> "the last 20 builds over all targets" I can only see
"the last five builds
>> for a particular builder". What's frustrating is that I'm
pretty sure the
>> buildbot is keeping the info, I just can't see how to get it.
>
> When you're on a builder's page (eg:
> http://lab.llvm.org:8011/builders/clang-x86_64-debian ) you can "view
> in waterfall" (
> http://lab.llvm.org:8011/waterfall?show=clang-x86_64-debian ) which
> gives you more history (even on the first page it's got about 14
> builds & it has multiple pages to go back further) but it's the
high
> detail waterfall view, not a simple red/green build status summary of
> each build.
>
> Having more history in the "recent builds" would be nice (or
somewhere
> else where that could be easily visualized)
Also, for what it's worth, the buildbot config can be changed to some
constant other than '5' for the recent builds list, but it would still
be only a constant across all builders. From the buildbot docs:

/builders/$BUILDERNAME
  This describes the given Builder and provides buttons to force a
build. A numbuilds=argument will control how many build lines are
displayed (5 by default).

Thanks. I'd made the wrong assumption "show in waterfall" was a
link back to
the multi-builder waterfall page. That's very helpful to know.

-----Original Message-----
From: David Blaikie [mailto:dblaikie at gmail.com] 
Sent: 12 November 2012 16:19
To: David Tweed
Cc: Renato Golin; Nico Weber; LLVM Developers Mailing List
Subject: Re: [LLVMdev] Need help reading the LLVM Buildbot results

On Mon, Nov 12, 2012 at 7:03 AM, David Tweed <david.tweed at arm.com>
wrote:
>> I don't believe so - I think only a certain sliding window of logs
are
>> kept (& I don't know of an easy way to view a particular
revision
>> across the bots either)
>
> For me a bigger issue has been inability to see a long list of results for
a
> particular builder (say, arm-cortex-a9-blah). Unfortunately while I can
see
> "the last 20 builds over all targets" I can only see "the
last five builds
> for a particular builder". What's frustrating is that I'm
pretty sure the
> buildbot is keeping the info, I just can't see how to get it.
When you're on a builder's page (eg:
http://lab.llvm.org:8011/builders/clang-x86_64-debian ) you can "view
in waterfall" (
http://lab.llvm.org:8011/waterfall?show=clang-x86_64-debian ) which
gives you more history (even on the first page it's got about 14
builds & it has multiple pages to go back further) but it's the high
detail waterfall view, not a simple red/green build status summary of
each build.

Having more history in the "recent builds" would be nice (or somewhere
else where that could be easily visualized)