llvm dev - Jan 2016 - [3.8 Release] RC1 has been tagged

Unfortunately I'm having lots of trouble with rc1 at this point:
* libcxxabi can't build, because it requires unwind.h, which we do not yet
have on FreeBSD 10.x (Ed Maste is working on it for 11.x, but that is not ready
for general consumption).
* The test-release.sh script has no option to disable only libcxxabi, you can
only disable libcxx, libcxxabi and libunwind together (maybe this can be
improved)
* Last time I hand-built libcxx, it still had a lot of test failures in the
locale parts, but I haven't had time to investigate.
* OpenMP does not support i386-freebsd, so I have to disable it there
* Last but not least: the host compiler on FreeBSD 10.x is clang 3.4.1 (the last
version that can build without C++11 support), and it crashes with a segfault
during building of CGBlocks.cpp.  I'll need to find some way to work around
this failure, since we cannot upgrade the compiler easily on FreeBSD 10.x.

I also had to hack the test-release.sh script to fix a number of problems that I
encountered during the 3.7.1 release, but haven't gotten to upstreaming
them.  E.g. the way the source code is checked out with symlinks all over the
place does not work, and I need to add a custom patch to clang-tools-extra to
make the tests succeed, because there is a race condition in the Makefile.

-Dimitry
> On 20 Jan 2016, at 11:30, Daniel Sanders <Daniel.Sanders at
imgtec.com> wrote:
> 
> This isn't rc1 but the tip of the release branch had some X86 test
failures on my most recent build:
> Failing Tests (24):
>    libc++ ::
std/input.output/file.streams/fstreams/filebuf.virtuals/overflow.pass.cpp
>    libc++ ::
std/input.output/file.streams/fstreams/filebuf.virtuals/underflow.pass.cpp
>    libc++ :: std/input.output/iostream.format/ext.manip/get_time.pass.cpp
>    libc++ :: std/input.output/iostream.format/ext.manip/put_time.pass.cpp
>    libc++ ::
std/input.output/iostreams.base/ios.base/ios.base.callback/register_callback.pass.cpp
>    libc++ ::
std/input.output/iostreams.base/ios.base/ios.base.locales/imbue.pass.cpp
>    libc++ ::
std/input.output/iostreams.base/ios/basic.ios.members/imbue.pass.cpp
>    libc++ ::
std/input.output/stream.buffers/streambuf/streambuf.cons/copy.pass.cpp
>    libc++ ::
std/input.output/stream.buffers/streambuf/streambuf.cons/default.pass.cpp
>    libc++ ::
std/input.output/stream.buffers/streambuf/streambuf.protected/streambuf.assign/assign.pass.cpp
>    libc++ ::
std/input.output/stream.buffers/streambuf/streambuf.protected/streambuf.assign/swap.pass.cpp
>    libc++ ::
std/localization/locale.categories/category.collate/locale.collate.byname/hash.pass.cpp
>    libc++ ::
std/localization/locale.categories/category.collate/locale.collate.byname/types.pass.cpp
>    libc++ ::
std/localization/locale.categories/category.ctype/locale.codecvt.byname/ctor_wchar_t.pass.cpp
>    libc++ ::
std/localization/locale.categories/category.ctype/locale.ctype.byname/types.pass.cpp
>    libc++ :: std/localization/locales/locale/locale.cons/default.pass.cpp
>    libc++ :: std/localization/locales/locale/locale.members/name.pass.cpp
>    libc++ :: std/localization/locales/locale/locale.operators/eq.pass.cpp
>    libc++ :: std/localization/locales/locale/locale.statics/global.pass.cpp
>    libc++ :: std/re/re.regex/re.regex.locale/imbue.pass.cpp
>    libc++ :: std/re/re.traits/default.pass.cpp
>    libc++ :: std/re/re.traits/getloc.pass.cpp
>    libc++ :: std/re/re.traits/imbue.pass.cpp
>    libomp :: barrier/omp_barrier.c
> 
> Big-endian mips gave this during phase 3:
> 	CMake Error at cmake/modules/HandleLLVMOptions.cmake:43 (message):
> 	  Host Clang must be able to find libstdc++4.7 or newer!
> It's possible that this is a machine config issue. We moved offices
over the weekend (just across the street) and my normal machine somehow lost the
ability to boot in the process. I'm borrowing a replacement at the moment.
> 
> Little-endian mips is just about to finish Phase 2 so I'll know if
it's a more general problem soon.
> 
>> -----Original Message-----
>> From: hwennborg at google.com [mailto:hwennborg at google.com] On
Behalf
>> Of Hans Wennborg
>> Sent: 19 January 2016 23:56
>> To: Ben Pope; Dimitry Andric; Daniel Sanders; Nikola Smiljanić; Brian
Cain;
>> Renato Golin; Sylvestre Ledru
>> Cc: cfe-dev; lldb-dev at lists.llvm.org; openmp-dev at lists.llvm.org;
llvm-dev
>> Subject: Re: [3.8 Release] RC1 has been tagged
>> 
>> (cc'ing non-legacy llvm-dev this time; apologies if you get this
>> twice. Please don't reply-all to the first one.)
>> 
>> On Tue, Jan 19, 2016 at 3:47 PM, Hans Wennborg <hans at
chromium.org>
>> wrote:
>>> Dear testers,
>>> 
>>> Start your engines; 3.8.0-rc1 was just tagged from the 3.8 branch
at
>>> r258223. (It took a little longer than I'd planned, sorry about
that.)
>>> 
>>> There are still a bunch of open merge requests and bug reports, but
I
>>> wanted to get the tag in so we can see what the build and test
status
>>> are on the various platforms.
>>> 
>>> I verified that it currently builds and tests cleanly for me on
x86_64
>>> Linux, Mac OS X* and Windows.
>>> 
>>> Please build, test, and upload binaries to the sftp. Let me know if
how it
>> goes.
>>> 
>>> Thanks,
>>> Hans
>>> 
>>> 
>>> [*] For Mac, I had to set CFLAGS="-isysroot `xcrun
-show-sdk-path`"
>>> CXXFLAGS="-isysroot `xcrun -show-sdk-path`" for the build
to work,
>>> otherwise stage-2 Clang couldn't find the SDK. I don't
remember if I
>>> had to do this last time; maybe some upgrade changed something.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: Message signed with OpenPGP using GPGMail
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160120/e720be85/attachment.sig>

On Wed, Jan 20, 2016 at 5:25 AM, Dimitry Andric <dimitry at andric.com>
wrote:
> Unfortunately I'm having lots of trouble with rc1 at this point:
> * libcxxabi can't build, because it requires unwind.h, which we do not
yet have on FreeBSD 10.x (Ed Maste is working on it for 11.x, but that is not
ready for general consumption).
> * The test-release.sh script has no option to disable only libcxxabi, you
can only disable libcxx, libcxxabi and libunwind together (maybe this can be
improved)
Yes, I'd be happy to take a patch for this, or I suppose you could
just hack it out locally when building.
> * Last time I hand-built libcxx, it still had a lot of test failures in the
locale parts, but I haven't had time to investigate.
Did we have that for 3.7 too?
> * OpenMP does not support i386-freebsd, so I have to disable it there
That's perfectly fine. Including OpenMP by default is new for this
release, so if it causes any problems, just exclude it.
> * Last but not least: the host compiler on FreeBSD 10.x is clang 3.4.1 (the
last version that can build without C++11 support), and it crashes with a
segfault during building of CGBlocks.cpp.  I'll need to find some way to
work around this failure, since we cannot upgrade the compiler easily on FreeBSD
10.x.
This sounds like the biggest problem. Is there a PR for the crash? I
suppose the alternatives are either to try not to tickle the crash in
our source, or fixing the 3.4.1 compiler?
> I also had to hack the test-release.sh script to fix a number of problems
that I encountered during the 3.7.1 release, but haven't gotten to
upstreaming them.  E.g. the way the source code is checked out with symlinks all
over the place does not work, and I need to add a custom patch to
clang-tools-extra to make the tests succeed, because there is a race condition
in the Makefile.
:-( What's not working with the symlinks? Please send patches. I'm
sorry for the trouble here.

Thanks,
Hans

On 20 Jan 2016, at 18:23, Hans Wennborg <hans at chromium.org>
wrote:
> 
> On Wed, Jan 20, 2016 at 5:25 AM, Dimitry Andric <dimitry at
andric.com> wrote:
>> Unfortunately I'm having lots of trouble with rc1 at this point:
>> * libcxxabi can't build, because it requires unwind.h, which we do
not yet have on FreeBSD 10.x (Ed Maste is working on it for 11.x, but that is
not ready for general consumption).
>> * The test-release.sh script has no option to disable only libcxxabi,
you can only disable libcxx, libcxxabi and libunwind together (maybe this can be
improved)
> 
> Yes, I'd be happy to take a patch for this, or I suppose you could
> just hack it out locally when building.
It's not terribly important right now, as libcxx isn't succeeding its
tests anyway.  I can locally hack it in, but I hope I won't forget it later.
:)

>> * Last time I hand-built libcxx, it still had a lot of test failures in
the locale parts, but I haven't had time to investigate.
> 
> Did we have that for 3.7 too?
With 3.7, we used autoconf builds for FreeBSD, and that skipped libcxx
altogether.  I did a few builds by hand, and I remember I saw similar failures. 
This is just something that nobody (from FreeBSD) has seriously looked at, and I
never had the time for it.  There are only so much hours in a day...

For example, recently with trunk r256945, I saw these:

Failing Tests (39):
    libc++ :: std/depr/depr.c.headers/stddef_h.pass.cpp
    libc++ :: std/depr/depr.c.headers/wchar_h.pass.cpp
    libc++ :: std/language.support/support.types/max_align_t.pass.cpp
    libc++ ::
std/localization/locale.categories/category.collate/locale.collate.byname/compare.pass.cpp
    libc++ ::
std/localization/locale.categories/category.collate/locale.collate.byname/transform.pass.cpp
    libc++ ::
std/localization/locale.categories/category.ctype/locale.ctype.byname/tolower_1.pass.cpp
    libc++ ::
std/localization/locale.categories/category.ctype/locale.ctype.byname/tolower_many.pass.cpp
    libc++ ::
std/localization/locale.categories/category.ctype/locale.ctype.byname/toupper_1.pass.cpp
    libc++ ::
std/localization/locale.categories/category.ctype/locale.ctype.byname/toupper_many.pass.cpp
    libc++ ::
std/localization/locale.categories/category.monetary/locale.money.get/locale.money.get.members/get_long_double_fr_FR.pass.cpp
    libc++ ::
std/localization/locale.categories/category.monetary/locale.money.get/locale.money.get.members/get_long_double_ru_RU.pass.cpp
    libc++ ::
std/localization/locale.categories/category.monetary/locale.money.get/locale.money.get.members/get_long_double_zh_CN.pass.cpp
    libc++ ::
std/localization/locale.categories/category.monetary/locale.money.put/locale.money.put.members/put_long_double_fr_FR.pass.cpp
    libc++ ::
std/localization/locale.categories/category.monetary/locale.money.put/locale.money.put.members/put_long_double_ru_RU.pass.cpp
    libc++ ::
std/localization/locale.categories/category.monetary/locale.money.put/locale.money.put.members/put_long_double_zh_CN.pass.cpp
    libc++ ::
std/localization/locale.categories/category.monetary/locale.moneypunct.byname/curr_symbol.pass.cpp
    libc++ ::
std/localization/locale.categories/category.monetary/locale.moneypunct.byname/grouping.pass.cpp
    libc++ ::
std/localization/locale.categories/category.monetary/locale.moneypunct.byname/neg_format.pass.cpp
    libc++ ::
std/localization/locale.categories/category.monetary/locale.moneypunct.byname/pos_format.pass.cpp
    libc++ ::
std/localization/locale.categories/category.monetary/locale.moneypunct.byname/thousands_sep.pass.cpp
    libc++ ::
std/localization/locale.categories/category.time/locale.time.get.byname/get_date.pass.cpp
    libc++ ::
std/localization/locale.categories/category.time/locale.time.get.byname/get_date_wide.pass.cpp
    libc++ ::
std/localization/locale.categories/category.time/locale.time.get.byname/get_one.pass.cpp
    libc++ ::
std/localization/locale.categories/category.time/locale.time.get.byname/get_one_wide.pass.cpp
    libc++ ::
std/localization/locale.categories/category.time/locale.time.put.byname/put1.pass.cpp
    libc++ ::
std/localization/locale.categories/facet.numpunct/locale.numpunct.byname/grouping.pass.cpp
    libc++ ::
std/localization/locale.categories/facet.numpunct/locale.numpunct.byname/thousands_sep.pass.cpp
    libc++ :: std/re/re.alg/re.alg.match/basic.pass.cpp
    libc++ :: std/re/re.alg/re.alg.match/ecma.pass.cpp
    libc++ :: std/re/re.alg/re.alg.match/extended.pass.cpp
    libc++ :: std/re/re.alg/re.alg.search/awk.pass.cpp
    libc++ :: std/re/re.alg/re.alg.search/basic.pass.cpp
    libc++ :: std/re/re.alg/re.alg.search/ecma.pass.cpp
    libc++ :: std/re/re.alg/re.alg.search/extended.pass.cpp
    libc++ :: std/re/re.traits/lookup_collatename.pass.cpp
    libc++ :: std/re/re.traits/transform_primary.pass.cpp
    libc++ :: std/re/re.traits/translate_nocase.pass.cpp
    libc++ :: std/strings/string.conversions/stof.pass.cpp
    libc++ ::
std/utilities/meta/meta.trans/meta.trans.other/aligned_storage.pass.cpp

Individual failures were typically of the form:

FAIL: libc++ ::
std/localization/locale.categories/category.monetary/locale.moneypunct.byname/thousands_sep.pass.cpp
(27654 of 30312)
******************** TEST 'libc++ ::
std/localization/locale.categories/category.monetary/locale.moneypunct.byname/thousands_sep.pass.cpp'
FAILED ********************
Compiled With: ['/usr/bin/clang++', '-o',
'/home/dim/obj/llvm-256945-trunk-freebsd11-i386-ninja-rel-1/projects/libcxx/test/std/localization/locale.categories/category.monetary/locale.moneypunct.byname/Output/thousands_sep.pass.cpp.o',
'-x', 'c++', '/share/dim/
src/llvm/trunk/projects/libcxx/test/std/localization/locale.categories/category.monetary/locale.moneypunct.byname/thousands_sep.pass.cpp',
'-c', '-v', '-std=c++1z', '-nostdinc++',
'-I/share/dim/src/llvm/trunk/projects/libcxx/test/support',
'-include', '/share/
dim/src/llvm/trunk/projects/libcxx/test/support/nasty_macros.hpp',
'-I/share/dim/src/llvm/trunk/projects/libcxx/include',
'&&', '/usr/bin/clang++', '-o',
'/home/dim/obj/llvm-256945-trunk-freebsd11-i386-ninja-rel-1/projects/libcxx/test/std/localization/locale.c
ategories/category.monetary/locale.moneypunct.byname/Output/thousands_sep.pass.cpp.exe',
'/home/dim/obj/llvm-256945-trunk-freebsd11-i386-ninja-rel-1/projects/libcxx/test/std/localization/locale.categories/category.monetary/locale.moneypunct.byname/Output/thous
ands_sep.pass.cpp.o', '-v', '-nodefaultlibs',
'-L/home/dim/obj/llvm-256945-trunk-freebsd11-i386-ninja-rel-1/lib',
'-Wl,-rpath,/home/dim/obj/llvm-256945-trunk-freebsd11-i386-ninja-rel-1/lib',
'-lc++', '-lc', '-lm', '-lpthread',
'-lgcc_s', '-lcxxrt']
Command:
['/home/dim/obj/llvm-256945-trunk-freebsd11-i386-ninja-rel-1/projects/libcxx/test/std/localization/locale.categories/category.monetary/locale.moneypunct.byname/Output/thousands_sep.pass.cpp.exe']
Exit Code: -6
Standard Error:
--
Assertion failed: (f.thousands_sep() == ' '), function main, file
/share/dim/src/llvm/trunk/projects/libcxx/test/std/localization/locale.categories/category.monetary/locale.moneypunct.byname/thousands_sep.pass.cpp,
line 103.
--

So there is probably quite some work to be done to get them all sorted out. 
(And localization stuff tends to be a tricky area, in my experience.)

>> * OpenMP does not support i386-freebsd, so I have to disable it there
> 
> That's perfectly fine. Including OpenMP by default is new for this
> release, so if it causes any problems, just exclude it.
Okay.  That said, in the past, I have built OpenMP for x86_64, and after some
twiddling, I remember that (most of) the tests ran correctly.  I will at least
try to do builds for x86_64.

>> * Last but not least: the host compiler on FreeBSD 10.x is clang 3.4.1
(the last version that can build without C++11 support), and it crashes with a
segfault during building of CGBlocks.cpp.  I'll need to find some way to
work around this failure, since we cannot upgrade the compiler easily on FreeBSD
10.x.
> 
> This sounds like the biggest problem. Is there a PR for the crash? I
> suppose the alternatives are either to try not to tickle the crash in
> our source, or fixing the 3.4.1 compiler?
There is no PR, but I did a bisection, and the crash turns out to be fixed by
r210467 ("[DAG] Expose NoSignedWrap, NoUnsignedWrap and Exact flags to
SelectionDAG.") by Andrea DiBiagio.

I do not know exactly *why* this fixes it, but this particular revision is
rather tricky to merge to a 3.4 tree, since it touches quite some parts of
SelectionDAG, and there were lots of stylistic changes and cleanups in this area
around (or before) that time.

I'm currently attempting to either modify the fix so it applies directly on
top of a 3.4 tree, or adding the other changes needed to be able to apply it. 
But it isn't trivial, unfortunately.

>> I also had to hack the test-release.sh script to fix a number of
problems that I encountered during the 3.7.1 release, but haven't gotten to
upstreaming them.  E.g. the way the source code is checked out with symlinks all
over the place does not work, and I need to add a custom patch to
clang-tools-extra to make the tests succeed, because there is a race condition
in the Makefile.
> 
> :-( What's not working with the symlinks? Please send patches. I'm
> sorry for the trouble here.
As I should have done earlier (after 3.7.1 in fact :), I committed the fix for
clang-tools-extra's test Makefile in r258327.  This is a trivial fix, so
please merge it to release_38.

As to the symlinks, the test-release.sh script originally checks out the sources
in parallel directories, e.g.:

llvm.src
cfe.src
compiler-rt.src

and so on.  Within llvm.src, symlinks are made to point to each of these.  For
some reason, on FreeBSD, this causes .cpp files under
llvm.src/tools/clang/tools/extra to not be able to find their include files, and
my log files show the following kind of errors:

/home/dim/llvm-3.7.0/rc3/llvm.src/tools/clang/tools/extra/clang-apply-replacements/lib/Tooling/../../include/clang-apply-replacements/Tooling/ApplyReplacements.h:19:10:
fatal error: 'clang/Tooling/Refactoring.h' file not found
#include "clang/Tooling/Refactoring.h"
         ^
1 error generated.

I remember trying lots of things to make this work, but failing.  Apparently
there is some issue with following a double symlink path, e.g.
llvm.src/tools/clang is a symlink to ../../cfe.src, while
llvm.src/tools/clang/tools/extra (which really is under ../../cfe.src/tools) is
a symlink to ../../clang-tools-extra.src.

The way I fixed this during the 3.7 test phase, is by changing test-release.sh
so it exports directly into these locations:

# Exporting llvm 3.7.0-rc3 sources to llvm.src
# Exporting cfe 3.7.0-rc3 sources to llvm.src/tools/clang
# Exporting clang-tools-extra 3.7.0-rc3 sources to
llvm.src/tools/clang/tools/extra
# Exporting compiler-rt 3.7.0-rc3 sources to llvm.src/projects/compiler-rt
# Exporting libcxx 3.7.0-rc3 sources to llvm.src/projects/libcxx
# Exporting libcxxabi 3.7.0-rc3 sources to llvm.src/projects/libcxxabi
# Exporting libunwind 3.7.0-rc3 sources to llvm.src/projects/libunwind
# Exporting test-suite 3.7.0-rc3 sources to llvm.src/projects/test-suite

This is probably not so handy if you want to create tarballs of the individual
components, though.

-Dimitry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: Message signed with OpenPGP using GPGMail
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160120/04ecb2c3/attachment.sig>