llvm dev - Dec 2018 - using emulated-tls on Darwin 8, 9, 10

Please excuse hobbiest-level question.

Darwin 11+ enables thread_local variables using system-level supports.

I have an interest in enabling TLS on darwin < 11  using emulated-tls. This
can be enabled with a few modest patches:

=========================--- a/include/llvm/ADT/Triple.h.orig	2018-10-02
17:38:10.000000000 -0700
+++ b/include/llvm/ADT/Triple.h	2018-10-02 17:38:58.000000000 -0700
@@ -682,7 +682,7 @@

 /// Tests whether the target uses emulated TLS as default.
 bool hasDefaultEmulatedTLS() const {
-    return isAndroid() || isOSOpenBSD() || isWindowsCygwinEnvironment();
+    return isAndroid() || isOSOpenBSD() || isWindowsCygwinEnvironment() ||
isMacOSXVersionLT(10, 7);
 }

 /// @}
=========================--- a/tools/clang/lib/CodeGen/ItaniumCXXABI.cpp.orig
2018-10-02 18:31:17.000000000 -0700
+++ b/tools/clang/lib/CodeGen/ItaniumCXXABI.cpp	2018-10-02 18:32:35.000000000
-0700
@@ -2255,7 +2255,7 @@
 const char *Name = "__cxa_atexit";
 if (TLS) {
   const llvm::Triple &T = CGF.getTarget().getTriple();
-    Name = T.isOSDarwin() ?  "_tlv_atexit" :
"__cxa_thread_atexit";
+    Name = (T.isOSDarwin() && !T.isMacOSXVersionLT(10, 7)) ? 
"_tlv_atexit" : "__cxa_thread_atexit";
 }

 // We're assuming that the destructor function is something we can
--- a/tools/clang/lib/Basic/Targets/OSTargets.h.orig	2018-10-02
17:14:10.000000000 -0700
+++ b/tools/clang/lib/Basic/Targets/OSTargets.h	2018-10-02 17:14:41.000000000
-0700
@@ -93,7 +93,7 @@
   this->TLSSupported = false;

   if (Triple.isMacOSX())
-      this->TLSSupported = !Triple.isMacOSXVersionLT(10, 7);
+      this->TLSSupported = !Triple.isMacOSXVersionLT(10, 4);
   else if (Triple.isiOS()) {
     // 64-bit iOS supported it from 8 onwards, 32-bit device from 9 onwards,
     // 32-bit simulator from 10 onwards.
=========================

and in doing so, when clang links against newer versions of libgcc ,
thread_local support is available. (In MacPorts, this is done using
-stdlib=macports-libstdc++, which links against a new version of libgcc from
gcc8).



I would like to enable this while using libc++ as well, however. 

To do this, libc++abi is built with cxa_thread_atexit.cpp added, and emutls.c is
built and used in libclang_rt.10.4.a.

However I am having an issue wherein the symbol ___emutls_get_address from
lilbclang_rt_10.4.a is not visible to libc++abi.dylib at runtime.


"___emutls_get_address"  is built into clang_rt:

$ nm /opt/local/libexec/llvm-5.0/lib/clang/5.0.1/lib/darwin/libclang_rt.10.4.a 
| grep emu
/opt/local/libexec/llvm-5.0/lib/clang/5.0.1/lib/darwin/libclang_rt.10.4.a(emutls.c.o):
0000000000000000 T ___emutls_get_address
00000000000001a0 t _emutls_init
0000000000000210 d _emutls_init_once.once
00000000000001c0 t _emutls_key_destructor
0000000000000220 d _emutls_mutex
00000000000002f8 b _emutls_num_object
0000000000000300 b _emutls_pthread_key

The objects appear to be included in the built executables (a simple example
using TLS 4.cpp made into "4"),  but the symbols are apparently not
visible to libc++abi.dylib at runtime.

$ nm 4 | grep emu
0000000100007070 t ___emutls_get_address
0000000100009480 d
___emutls_v._ZGVZL18thread_with_accessPvE23counting_function_local
0000000100009400 d ___emutls_v._ZL15counting_static
0000000100009420 d ___emutls_v._ZN12_GLOBAL__N_128counting_anonymous_namespaceE
0000000100009460 d
___emutls_v._ZZL18thread_with_accessPvE23counting_function_local
00000001000094a0 d ___emutls_v.__tls_guard
0000000100009440 d ___emutls_v.counting_extern
0000000100007200 t _emutls_init
00000001000094c0 d _emutls_init_once.once
0000000100007220 t _emutls_key_destructor
00000001000094d0 d _emutls_mutex
0000000100009538 b _emutls_num_object
0000000100009540 b _emutls_pthread_key


$ ./4
info: testing pthread_create
dyld: lazy symbol binding failed: Symbol not found: ___emutls_get_address
 Referenced from: /usr/lib/libc++abi.dylib
 Expected in: flat namespace

dyld: Symbol not found: ___emutls_get_address
 Referenced from: /usr/lib/libc++abi.dylib
 Expected in: flat namespace

Trace/BPT trap


Adding -Wl,-flat_namespace does not make the symbols visible.


If I try to fix it like this by forcing the symbol to be exported, I get a clue:

$ clang++ -std=c++11 -stdlib=libc++ -Wl,-exported_symbol,___emutls_get_address 
-o 4 4.cpp
ld: warning: cannot export hidden symbol ___emutls_get_address from
/opt/local/libexec/llvm-5.0/lib/clang/5.0.1/lib/darwin/libclang_rt.osx.a(emutls.c.o)


Things do work apparently properly however if I build emutls.c directly into
libc++abi.dylib, using a slightly modified version of libcxxabi that includes
the emutls.c source file in the build:

$ nm libc++abi.kenspecial.20181206.dylib | grep emutls_
0000000000020054 T ___emutls_get_address
0000000000025458 s ___emutls_t._ZN10__cxxabiv112_GLOBAL__N_15dtorsE
000000000002a270 d ___emutls_v._ZN10__cxxabiv112_GLOBAL__N_111dtors_aliveE
000000000002a290 d ___emutls_v._ZN10__cxxabiv112_GLOBAL__N_15dtorsE
000000000002a2b0 d _emutls_get_index.once
00000000000201c8 t _emutls_init
00000000000201ea t _emutls_key_destructor
000000000002a2c0 d _emutls_mutex
000000000002a360 b _emutls_num_object
000000000002a368 b _emutls_pthread_key


and with this modified libc++abi.dylib, all the symbols are found, things work
correctly, and all the tests that should pass in the llvm "testit"
suite do pass.


I presume that the visibility of ___emutls_get_address in libclang_rt.osx.a is
being controlled by some mechanism (apple_versioning.c ?).

I notice that the library is built with -fvisibility=hidden

1) is my approach here generally reasonable?

2) is there a toggle to allow the ___emutls_get_address symbol in 
libclang_rt.10.4.a to be made visible to libc++abi.dylib ?

Best,

Ken

On 2018-12-07 22:30, Ken Cunningham via llvm-dev wrote:
> Please excuse hobbiest-level question.
> 
> Darwin 11+ enables thread_local variables using system-level supports.
> 
> I have an interest in enabling TLS on darwin < 11  using emulated-tls.
Is anyone still running macOS 10.6 or older?

-- 
/Jacob Carlborg

> On 2018-12-07 22:30, Ken Cunningham via llvm-dev wrote:
>> Please excuse hobbiest-level question.
>> Darwin 11+ enables thread_local variables using system-level supports.
>> I have an interest in enabling TLS on darwin < 11  using
emulated-tls.
> 
> Is anyone still running macOS 10.6 or older?
> 
> -- 
> /Jacob Carlborg
> 

[off topic, apologies] 
Yes, there remains a surprising amount of interest in these systems. MacPorts
sees very frequent user tickets about them, many tickets lately about
thread_local. All versions of clang/llvm run on 10.5+ Intel systems with minor
tweaks, including the latest trunk versions, and  this enables almost all
current opensource software to be used on these systems. Software (like
browsers) geared to these systems see downloads in the low thousands per
release. I guess people either have a need for these systems for some software
version or workflow, or have otherwise workable hardware that as been obsoleted
by newer os requirements.

Even PowerPC systems get frequent tickets at MacPorts, and we support them as we
can. Some versions of clang will run on PowerPC Macs with workable coverage (in
fact the only remaining hole is a persisting issue with a bad address for c++
exceptions being supplied).

[/on topic]



Making the emutls objects in libclang_rt visible to libcxxabi.dylib at runtime
has been a hiccup, tho.

It works but seems a shame to hack emutls.c into the libc++abi code when the
objects are already sitting there in the executable, but (AFAICT) have the wrong
visibility setting to allow use.

At present I don't seem to know enough about the underpinnings to see how to
change the visibility settings on the emutls objects from libclang_rt when
placed into the executable.

Appreciate any thoughts on this, thanks for your interest, and sorry for the
slight divergence above.

Ken (MacPorts dev)