llvm dev - Nov 2021 - Exceptions not working when cross compiling for ARM Cortex M4 with clang and precompiled libraries from ARM GNU GCC Toolchain

Hello!

I have been struggling with enabling exceptions for the Clang build, when
cross compiling using precompiled libraries from ARM GNU GCC Toolchain.

LLVM version: 13.0.0
CMake version: 3.21.3
targeted CPU: STM32L432KC, ARM Cortex M4

The C++ flags:

-mthumb -mcpu=cortex-m4 -mfloat-abi=hard -mfpu=fpv4-sp-d16
-nodefaultlibs
--sysroot=${ARM_GNU_TOOLCHAIN_PATH}/arm-none-eabi
-flto
-fdata-sections -ffunction-sections
# For <iostream>, <string>, ...
-isystem
"${ARM_GNU_TOOLCHAIN_PATH}/arm-none-eabi/include/c++/${ARM_GNU_TOOLCHAIN_GCC_VERSION}/"
# For <bits/*>, ...
-isystem
"${ARM_GNU_TOOLCHAIN_PATH}/arm-none-eabi/include/c++/${ARM_GNU_TOOLCHAIN_GCC_VERSION}/arm-none-eabi/thumb/v7e-m+fp/hard/"
-fexceptions
-g


ARM_GNU_TOOLCHAIN_PATH is the root path to the mentioned ARM GNU GCC
Toolchain. ARM_GNU_TOOLCHAIN_GCC_VERSION is equal to 10.3.1. One can
download it here:
https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-rm/downloads

The linker flags:

-mthumb -mcpu=cortex-m4 -mfloat-abi=hard -mfpu=fpv4-sp-d16
-nodefaultlibs
--sysroot=${ARM_GNU_TOOLCHAIN_PATH}/arm-none-eabi
-flto
-fdata-sections -ffunction-sections
-Wl,--gc-sections
-g
-flto
-fexceptions
# Path to standard libraries: libc, libm, ...
-L"${ARM_GNU_TOOLCHAIN_PATH}/arm-none-eabi/lib/thumb/v7e-m+fp/hard/"
# Path to libgcc
-L"${ARM_GNU_TOOLCHAIN_PATH}/lib/gcc/arm-none-eabi/${ARM_GNU_TOOLCHAIN_GCC_VERSION}/thumb/v7e-m+fp/hard/"
-lc -lm -lnosys -lstdc++ -lgcc")


I use custom linker script and custom startup file to compile the firmware
properly. I attach them both.

This code:

try
{
    throw std::runtime_error{"Some error!"};
} catch (const std::exception&e)
{
    printf("Error: %s\r\n", e.what());
}

Will not work for the Clang build. It will crash. It works normally with
the GCC build.

The problem is that clang generates ".got" section, which it inputs
automatically, without being "said", to RAM. I have included it in the
".data" section, because otherwise it breaks the global variable
initialization within the startup code. GOT section is normally related to
dynamic symbol resolution. Since I link everything statically, I don't
understand why this section shows up there.

The mentioned ".got" section contains one variable, which is an
address
pointing to some place in "rodata". Unfortunately, this address is
wrong.
It points to some random rodata, where some ASCII strings are held. It
shall point to the symbol:
vtable for __cxxabiv1::__class_type_info + 8 in section .rodata
I know that, because I have scrapped it from the GCC build (performed with
the downloaded toolchain), for which the exceptions work properly.

The code crashes in the __cxa_type_match:
0x080096d6 in __cxa_type_match ()
─── Assembly ─────────────
 0x080096d6  ? ldr    r3, [r4, #0]                # r4 holds the address of
"got"
 0x080096d8  ? mov    r1, r6
 0x080096da  ? mov    r0, r4
 0x080096dc  ? ldr    r6, [r3, #16]              # Loaded from the address
pointed by (r3 + 16)
 0x080096de  ? add    r2, sp, #4
 0x080096e0  ? movs    r3, #1
 0x080096e2  ? blx    r6                            # This line crashes

According to the comments in the assembly:
r4 contains the address to the got section, which is 0x20000000. Under
0x20000000 lies the address to the random ASCII rodata. It means that:

    ldr    r6, [r3, #16]

Loads the ASCII characters to r6. Then we perform the blx jump with
contents of r6, which is wrong, because ASCII data is no address.

Using gdb I am able to write data to RAM by hand. I have found the address
of the symbol vtable for __cxxabiv1::__class_type_info + 8 in section
.rodata, which is 0x8019004. Then I wrote this address under 0x20000000
(the "got" section). Unfortunately, the program didn't end up as
expected -
within the "catch" block, but went to "_exit", through
"std::terminate",
what means that no suitable handler has been found during stack unwinding.
The stack trace from that attempt:

[0] from 0x08008df0 in _exit
[1] from 0x08008d42 in abort
[2] from 0x08009d08 in __gnu_cxx::__verbose_terminate_handler()
[3] from 0x08008cc2 in __cxxabiv1::__terminate(void (*)())
[4] from 0x08008cf2 in std::terminate()
[5] from 0x08009c08 in __cxa_throw
[6] from 0x0800fedc in test_throws_uncaught_exception()+52 at
/home/kacper/Workspace/Aura/tests/device/test_serial_logger/test_logs_uncaught_exception.cpp:11
[7] from 0x08014ff0 in run_test(void (*)(), char const*, unsigned int)+92
at test_logs_uncaught_exception_runner.cpp:62
[8] from 0x08014f8e in test_main()+12 at
test_logs_uncaught_exception_runner.cpp:79
[9] from 0x08014440 in app_main::$_0::operator()() const+16 at
/home/kacper/Workspace/Aura/tests/device/setup/test_runner.cpp:24

I attach section dump and disassembly of the program as well.

Am I missing some compiler or linker flag?

I am fighting with it for a week at least. I was hoping that I will be able
to push my project with clang build as the main build, but I guess I will
have to fall back to GCC instead.

Any help would be appreciated.

Kind regards,
Kacper Kowalski
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211120/bc970e7f/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: startup_stm32l432xx.s
Type: application/octet-stream
Size: 12026 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211120/bc970e7f/attachment-0004.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: STM32L432KCUx_FLASH.ld
Type: application/octet-stream
Size: 6170 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211120/bc970e7f/attachment-0005.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_logs_uncaught_exception.objdump
Type: application/octet-stream
Size: 1683167 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211120/bc970e7f/attachment-0006.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_logs_uncaught_exception.disass
Type: application/octet-stream
Size: 2396030 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211120/bc970e7f/attachment-0007.obj>

Some small suggestions:

LTO makes everything more complicated; make sure things work without -flto
before trying LTO.

A “.got” section shows up when an object file compiled with -fPIC/-fPIE accesses
an external global variable or constant.  You can check if an object file
contains a relocation like this with objdump -r (something like R_ARM_GOT_PREL).
For a static link, the linker should fix it to point to the right address. If
the linker isn’t computing the correct address, my best guess is that something
is wrong with the linker script.  Are you setting the address of flash/RAM
correctly?  Is the difference between the address in the GOT and the actual
address related to the size of some section? (The consequences of getting this
wrong might not be immediately obvious if the code is mostly using pc-relative
addressing.)

If you have a working build with gcc, you might want to compare to see what,
exactly, is different.  Check the link line with “-###”; are you using the same
linker?  The same libraries?

-Eli

From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of Kacper
Kowalski via llvm-dev
Sent: Saturday, November 20, 2021 6:22 AM
To: llvm-dev <llvm-dev at lists.llvm.org>
Subject: [llvm-dev] Exceptions not working when cross compiling for ARM Cortex
M4 with clang and precompiled libraries from ARM GNU GCC Toolchain


WARNING: This email originated from outside of Qualcomm. Please be wary of any
links or attachments, and do not enable macros.
Hello!

I have been struggling with enabling exceptions for the Clang build, when cross
compiling using precompiled libraries from ARM GNU GCC Toolchain.

LLVM version: 13.0.0
CMake version: 3.21.3
targeted CPU: STM32L432KC, ARM Cortex M4
The C++ flags:

-mthumb -mcpu=cortex-m4 -mfloat-abi=hard -mfpu=fpv4-sp-d16

-nodefaultlibs

--sysroot=${ARM_GNU_TOOLCHAIN_PATH}/arm-none-eabi

-flto

-fdata-sections -ffunction-sections

# For <iostream>, <string>, ...

-isystem
"${ARM_GNU_TOOLCHAIN_PATH}/arm-none-eabi/include/c++/${ARM_GNU_TOOLCHAIN_GCC_VERSION}/"

# For <bits/*>, ...

-isystem
"${ARM_GNU_TOOLCHAIN_PATH}/arm-none-eabi/include/c++/${ARM_GNU_TOOLCHAIN_GCC_VERSION}/arm-none-eabi/thumb/v7e-m+fp/hard/"

-fexceptions

-g

ARM_GNU_TOOLCHAIN_PATH is the root path to the mentioned ARM GNU GCC Toolchain.
ARM_GNU_TOOLCHAIN_GCC_VERSION is equal to 10.3.1. One can download it here:
https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-rm/downloads

The linker flags:


-mthumb -mcpu=cortex-m4 -mfloat-abi=hard -mfpu=fpv4-sp-d16

-nodefaultlibs

--sysroot=${ARM_GNU_TOOLCHAIN_PATH}/arm-none-eabi

-flto

-fdata-sections -ffunction-sections

-Wl,--gc-sections

-g

-flto

-fexceptions

# Path to standard libraries: libc, libm, ...

-L"${ARM_GNU_TOOLCHAIN_PATH}/arm-none-eabi/lib/thumb/v7e-m+fp/hard/"

# Path to libgcc

-L"${ARM_GNU_TOOLCHAIN_PATH}/lib/gcc/arm-none-eabi/${ARM_GNU_TOOLCHAIN_GCC_VERSION}/thumb/v7e-m+fp/hard/"

-lc -lm -lnosys -lstdc++ -lgcc")

I use custom linker script and custom startup file to compile the firmware
properly. I attach them both.

This code:

try

{

    throw std::runtime_error{"Some error!"};

} catch (const std::exception&e)

{

    printf("Error: %s\r\n", e.what());

}
Will not work for the Clang build. It will crash. It works normally with the GCC
build.

The problem is that clang generates ".got" section, which it inputs
automatically, without being "said", to RAM. I have included it in the
".data" section, because otherwise it breaks the global variable
initialization within the startup code. GOT section is normally related to
dynamic symbol resolution. Since I link everything statically, I don't
understand why this section shows up there.

The mentioned ".got" section contains one variable, which is an
address pointing to some place in "rodata". Unfortunately, this
address is wrong. It points to some random rodata, where some ASCII strings are
held. It shall point to the symbol:
vtable for __cxxabiv1::__class_type_info + 8 in section .rodata
I know that, because I have scrapped it from the GCC build (performed with the
downloaded toolchain), for which the exceptions work properly.

The code crashes in the __cxa_type_match:
0x080096d6 in __cxa_type_match ()
─── Assembly ─────────────
 0x080096d6  ? ldr    r3, [r4, #0]                # r4 holds the address of
"got"
 0x080096d8  ? mov    r1, r6
 0x080096da  ? mov    r0, r4
 0x080096dc  ? ldr    r6, [r3, #16]              # Loaded from the address
pointed by (r3 + 16)
 0x080096de  ? add    r2, sp, #4
 0x080096e0  ? movs    r3, #1
 0x080096e2  ? blx    r6                            # This line crashes

According to the comments in the assembly:
r4 contains the address to the got section, which is 0x20000000. Under
0x20000000 lies the address to the random ASCII rodata. It means that:
    ldr    r6, [r3, #16]

Loads the ASCII characters to r6. Then we perform the blx jump with contents of
r6, which is wrong, because ASCII data is no address.

Using gdb I am able to write data to RAM by hand. I have found the address of
the symbol vtable for __cxxabiv1::__class_type_info + 8 in section .rodata,
which is 0x8019004. Then I wrote this address under 0x20000000 (the
"got" section). Unfortunately, the program didn't end up as
expected - within the "catch" block, but went to "_exit",
through "std::terminate", what means that no suitable handler has been
found during stack unwinding. The stack trace from that attempt:

[0] from 0x08008df0 in _exit
[1] from 0x08008d42 in abort
[2] from 0x08009d08 in __gnu_cxx::__verbose_terminate_handler()
[3] from 0x08008cc2 in __cxxabiv1::__terminate(void (*)())
[4] from 0x08008cf2 in std::terminate()
[5] from 0x08009c08 in __cxa_throw
[6] from 0x0800fedc in test_throws_uncaught_exception()+52 at
/home/kacper/Workspace/Aura/tests/device/test_serial_logger/test_logs_uncaught_exception.cpp:11
[7] from 0x08014ff0 in run_test(void (*)(), char const*, unsigned int)+92 at
test_logs_uncaught_exception_runner.cpp:62
[8] from 0x08014f8e in test_main()+12 at
test_logs_uncaught_exception_runner.cpp:79
[9] from 0x08014440 in app_main::$_0::operator()() const+16 at
/home/kacper/Workspace/Aura/tests/device/setup/test_runner.cpp:24

I attach section dump and disassembly of the program as well.

Am I missing some compiler or linker flag?

I am fighting with it for a week at least. I was hoping that I will be able to
push my project with clang build as the main build, but I guess I will have to
fall back to GCC instead.

Any help would be appreciated.

Kind regards,
Kacper Kowalski
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211122/6aeff87d/attachment-0001.html>