llvm dev - May 2016 - llvm-toolchain-3.8 on lower arm targets, specifically Debian armel and Raspbian.

llvm-toolchain-3.8 seems to have problems on debian armel and raspbian.

On raspbian it builds but our armv7 contamination checker blocked it 
from entering the repo. Further investigation showed that
"compiler-rt"
was being built with -march=armv7 . I was able to remove the -march with 
some build-system hacker but then I got a failure on
projects/compiler-rt/lib/builtins/arm/sync_fetch_and_add_4.S
> [  8%] Building ASM object 
>
projects/compiler-rt/lib/builtins/CMakeFiles/clang_rt.builtins-armhf.dir/arm/sync_fetch_and_add_4.S.o
> cd 
> /llvm-toolchain-3.8-3.8/build-llvm/projects/compiler-rt/lib/builtins 
> && /usr/bin/gcc-5  -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS 
> -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS 
> -I/llvm-toolchain-3.8-3.8/build-llvm/projects/compiler-rt/lib/builtins 
> -I/llvm-toolchain-3.8-3.8/projects/compiler-rt/lib/builtins 
> -I/llvm-toolchain-3.8-3.8/build-llvm/include 
> -I/llvm-toolchain-3.8-3.8/include  -fPIC -O2 -g -DNDEBUG    
> -mfloat-abi=hard -o 
> CMakeFiles/clang_rt.builtins-armhf.dir/arm/sync_fetch_and_add_4.S.o -c 
>
/llvm-toolchain-3.8-3.8/projects/compiler-rt/lib/builtins/arm/sync_fetch_and_add_4.S
>
/llvm-toolchain-3.8-3.8/projects/compiler-rt/lib/builtins/arm/sync_fetch_and_add_4.S:
> Assembler messages:
>
/llvm-toolchain-3.8-3.8/projects/compiler-rt/lib/builtins/arm/sync_fetch_and_add_4.S:20:
> Error: selected processor does not support `dmb' in Thumb mode
>
/llvm-toolchain-3.8-3.8/projects/compiler-rt/lib/builtins/arm/sync_fetch_and_add_4.S:20:
> Error: selected processor does not support `ldrex r0,[r12]' in Thumb
mode
>
/llvm-toolchain-3.8-3.8/projects/compiler-rt/lib/builtins/arm/sync_fetch_and_add_4.S:20:
> Error: cannot honor width suffix -- `add r2,r0,r1'
>
/llvm-toolchain-3.8-3.8/projects/compiler-rt/lib/builtins/arm/sync_fetch_and_add_4.S:20:
> Error: selected processor does not support `strex r3,r2,[r12]' in 
> Thumb mode
>
/llvm-toolchain-3.8-3.8/projects/compiler-rt/lib/builtins/arm/sync_fetch_and_add_4.S:20:
> Error: selected processor does not support `dmb' in Thumb mode
>
projects/compiler-rt/lib/builtins/CMakeFiles/clang_rt.builtins-armhf.dir/build.make:1097:
> recipe for target 
>
'projects/compiler-rt/lib/builtins/CMakeFiles/clang_rt.builtins-armhf.dir/arm/sync_fetch_and_add_4.S.o'
> failed
> make[4]: *** 
>
[projects/compiler-rt/lib/builtins/CMakeFiles/clang_rt.builtins-armhf.dir/arm/sync_fetch_and_add_4.S.o]
> Error 1
> make[4]: Leaving directory '/llvm-toolchain-3.8-3.8/build-llvm'
> CMakeFiles/Makefile2:12146: recipe for target 
>
'projects/compiler-rt/lib/builtins/CMakeFiles/clang_rt.builtins-armhf.dir/all'
> failed
> make[3]: *** 
>
[projects/compiler-rt/lib/builtins/CMakeFiles/clang_rt.builtins-armhf.dir/all]
> Error 2
> make[3]: *** Waiting for unfinished jobs...
I traced this back to code in compiler-rt/lib/builtins/arm/sync-ops.h . 
The comments talk about both arm mode and thumb2 mode code but I only 
actually see thumb2 code in the file. I'm kinda stuck here.

I expect this would be a problem for armel too (though it may go 
unnoticed due to the fact the armel autobuilders now run on armv7 
hardware and afaict Debian armel unlike raspbian doesn't have any 
contamination checking in place) but the build doesn't even get that 
far. It fails earlier with.
> /«PKGBUILDDIR»/include/llvm/Support/ThreadPool.h: In member function
'std::shared_future<void>  llvm::ThreadPool::async(Function&&,
Args&&  ...)':
> /«PKGBUILDDIR»/include/llvm/Support/ThreadPool.h:77:77: error: return type
'class std::shared_future<void>' is incomplete
>     inline std::shared_future<VoidTy>  async(Function&&F,
Args&&... ArgList) {
>                                                                            
^
> /«PKGBUILDDIR»/include/llvm/Support/ThreadPool.h: In member function
'std::shared_future<void> 
llvm::ThreadPool::async(Function&&)':
> /«PKGBUILDDIR»/include/llvm/Support/ThreadPool.h:95:57: error: return type
'class std::shared_future<void>' is incomplete
>     inline std::shared_future<VoidTy>  async(Function&&F) {

Hi Peter,

On 16 May 2016 at 22:25, peter green via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> I traced this back to code in compiler-rt/lib/builtins/arm/sync-ops.h . The
> comments talk about both arm mode and thumb2 mode code but I only actually
> see thumb2 code in the file. I'm kinda stuck here.
Yes, it looks like we'd need to conditionally compile these functions
in ARM mode and use the v6 barrier instead of dmb ("mcr p15, #0, r0,
c7, c10, #5" I believe) to support the ARM1176JZF-S in RPi. You'd
probably also want the build system to use an explicit -march=armv6 or
something so you're not at the mercy of how the host compiler was
configured.

I suspect you're the first person to try and get compiler-rt going for
pre-v7 hardware, which most of us prefer to pretend never happened
most of the time.
> I expect this would be a problem for armel too (though it may go unnoticed
> due to the fact the armel autobuilders now run on armv7 hardware and afaict
> Debian armel unlike raspbian doesn't have any contamination checking in
> place) but the build doesn't even get that far. It fails earlier with.
I'm not even sure where we'd begin for compiler-rt on armel, which
looks like it's supposed to work back to armv4t. That version doesn't
even *have* basic synchronization primitives, though I think I heard
the Linux kernel emulates them at some special restartable address?

The first step would probably be to find out where that is and make
compiler-rt (and/or LLVM directly) use it.
>> /«PKGBUILDDIR»/include/llvm/Support/ThreadPool.h: In member function
>> 'std::shared_future<void> 
llvm::ThreadPool::async(Function&&, Args&&
>> ...)':
I think I encountered similar recently: as I recall libstdc++ just
doesn't support the C++11 thread features LLVM needs on such an old
architecture. I think you'd have to add support before you could build
LLVM (or maybe bootstrap libc++ support). It's likely to be a hard
road either way.

Cheers.

Tim.

On 5/17/16 11:07 AM, Tim Northover via llvm-dev wrote:
> Hi Peter,
>
> On 16 May 2016 at 22:25, peter green via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>> I traced this back to code in compiler-rt/lib/builtins/arm/sync-ops.h .
The
>> comments talk about both arm mode and thumb2 mode code but I only
actually
>> see thumb2 code in the file. I'm kinda stuck here.
>
> Yes, it looks like we'd need to conditionally compile these functions
> in ARM mode and use the v6 barrier instead of dmb ("mcr p15, #0, r0,
> c7, c10, #5" I believe) to support the ARM1176JZF-S in RPi. You'd
> probably also want the build system to use an explicit -march=armv6 or
> something so you're not at the mercy of how the host compiler was
> configured.
>
> I suspect you're the first person to try and get compiler-rt going for
> pre-v7 hardware, which most of us prefer to pretend never happened
> most of the time.
I am bad, and I should feel bad... I had it working for v4/v4t and v6m, 
but then never set up a buildbot to make sure it stayed working :/ (also 
may have failed to upstream the (admittedly simple) patches for it).
>
>> I expect this would be a problem for armel too (though it may go
unnoticed
>> due to the fact the armel autobuilders now run on armv7 hardware and
afaict
>> Debian armel unlike raspbian doesn't have any contamination
checking in
>> place) but the build doesn't even get that far. It fails earlier
with.
>
> I'm not even sure where we'd begin for compiler-rt on armel, which
> looks like it's supposed to work back to armv4t. That version
doesn't
> even *have* basic synchronization primitives, though I think I heard
> the Linux kernel emulates them at some special restartable address?
Yes, it does:

   typedef int (__kernel_cmpxchg_t) (int old, int new, int *ptr);
   #define __kernel_cmpxchg (*(__kernel_cmpxchg_t *) 0xffff0fc0)

   typedef void (__kernel_dmb_t) (void);
   #define __kernel_dmb (*(__kernel_dmb_t *) 0xffff0fa0)

Though there is still an open question of what to do for baremetal, 
which of course doesn't have a kernel (though that's a separate issue 
from Peter's problems here).


Jon
>
> The first step would probably be to find out where that is and make
> compiler-rt (and/or LLVM directly) use it.
>
>>> /«PKGBUILDDIR»/include/llvm/Support/ThreadPool.h: In member
function
>>> 'std::shared_future<void> 
llvm::ThreadPool::async(Function&&, Args&&
>>> ...)':
>
> I think I encountered similar recently: as I recall libstdc++ just
> doesn't support the C++11 thread features LLVM needs on such an old
> architecture. I think you'd have to add support before you could
> build LLVM (or maybe bootstrap libc++ support). It's likely to be a
> hard road either way.
 >
> Cheers.
>
> Tim.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-- 
Jon Roelofs
jonathan at codesourcery.com
CodeSourcery / Mentor Embedded

On 17/05/16 18:07, Tim Northover wrote:
> Yes, it looks like we'd need to conditionally compile these functions
> in ARM mode and use the v6 barrier instead of dmb ("mcr p15, #0, r0,
> c7, c10, #5" I believe) to support the ARM1176JZF-S in RPi. You'd
> probably also want the build system to use an explicit -march=armv6 or
> something so you're not at the mercy of how the host compiler was
> configured.
>
> I suspect you're the first person to try and get compiler-rt going for
> pre-v7 hardware, which most of us prefer to pretend never happened
> most of the time.
>    
Thank you very much for your reply.

Can you clarify what exactly compiler-rt is and how important it is to 
the stack as a whole. It seems that llvm-toolchain-3.7 built 
successfully on raspbian without building compiler-rt while 
llvm-toolchain-3.8 built an armv7 compiler-rt (and hence got blocked 
from entering the raspbian repos).