Hi all, I'm trying to understand why LLVM uses @rpath as the install name in its dynamic libraries on macOS. This complicates the process of linking against libLLVM from third-party tools and isn't nearly as common a practice on macOS as it is on Linux. I tried tracing through history on the LLVM repo and the main thing I could find was a commit from ages ago saying that the libraries are now relocatable. While that's indeed true, rpaths put a burden on all consumers of the library (who now need to pass special flags to their linker) and unless there's a pressing need for them, most libraries avoid the practice. Indeed, the use of @rpath in LLVM seems to have been a bit confusing and has undergone dozens of changes over time, with reverts, changes to and from @executable_path (a related but different relative path inside the macOS loader), and so on. Here's the commit that seemed to introduce it: https://github.com/llvm-mirror/llvm/commit/fb24ccfa32c12dc7ab12b70f2c3db404dbab7581 Here's one of the reverts of the reverts: https://github.com/llvm-mirror/llvm/commit/fb24ccfa32c12dc7ab12b70f2c3db404dbab7581 Here's the switch to @executable_path: https://github.com/llvm-mirror/llvm/commit/692c94c1c9b5642cecb8fc7b6fa5ce3145b70ff1 Here it is switching back to @rpath: https://github.com/llvm-mirror/llvm/commit/37c04a0d4d0df0e2efddcc76a1036a9fc384e61a And here's an unanswered question about why it's there in the first place: http://lists.llvm.org/pipermail/llvm-dev/2016-January/094463.html Along with a downstream issue that needed to work around it: https://github.com/Homebrew/legacy-homebrew/issues/47149 And another downstream issue that outright patches out LLVM's @rpath support (authored by me): https://github.com/NixOS/nixpkgs/pull/30150 Anyway, I'm curious what the use case is for a relocatable libLLVM.dylib at the cost of downstream confusion. If it is indeed essential, could it be made an option in the build so that those of us who don't need relocatable LLVM libraries don't need to hack at the build files to turn it off? Thanks, Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171009/5365f92f/attachment.html>
Apologies, I appear to have pasted the same link twice in my original email. This is the actual commit that introduced the @rpath feature: https://github.com/llvm-mirror/llvm/commit/c94f3ae0278dc781cd60051790bf8a8003a13873 On Mon, Oct 9, 2017 at 10:24 AM, Daniel Peebles <pumpkingod at gmail.com> wrote:> Hi all, > > I'm trying to understand why LLVM uses @rpath as the install name in its > dynamic libraries on macOS. This complicates the process of linking against > libLLVM from third-party tools and isn't nearly as common a practice on > macOS as it is on Linux. I tried tracing through history on the LLVM repo > and the main thing I could find was a commit from ages ago saying that the > libraries are now relocatable. While that's indeed true, rpaths put a > burden on all consumers of the library (who now need to pass special flags > to their linker) and unless there's a pressing need for them, most > libraries avoid the practice. Indeed, the use of @rpath in LLVM seems to > have been a bit confusing and has undergone dozens of changes over time, > with reverts, changes to and from @executable_path (a related but different > relative path inside the macOS loader), and so on. > > Here's the commit that seemed to introduce it: https://github.com/llvm- > mirror/llvm/commit/fb24ccfa32c12dc7ab12b70f2c3db404dbab7581 > Here's one of the reverts of the reverts: https://github.com/ > llvm-mirror/llvm/commit/fb24ccfa32c12dc7ab12b70f2c3db404dbab7581 > Here's the switch to @executable_path: https:// > github.com/llvm-mirror/llvm/commit/692c94c1c9b5642cecb8fc7b6fa5ce > 3145b70ff1 > Here it is switching back to @rpath: https://github.com/ > llvm-mirror/llvm/commit/37c04a0d4d0df0e2efddcc76a1036a9fc384e61a > > And here's an unanswered question about why it's there in the first place: > http://lists.llvm.org/pipermail/llvm-dev/2016-January/094463.html > Along with a downstream issue that needed to work around it: > https://github.com/Homebrew/legacy-homebrew/issues/47149 > And another downstream issue that outright patches out LLVM's @rpath > support (authored by me): https://github.com/NixOS/nixpkgs/pull/30150 > > Anyway, I'm curious what the use case is for a relocatable libLLVM.dylib > at the cost of downstream confusion. If it is indeed essential, could it be > made an option in the build so that those of us who don't need relocatable > LLVM libraries don't need to hack at the build files to turn it off? > > Thanks, > Dan >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171009/7fc24570/attachment-0001.html>
Can you please describe the actual problem you're trying to fix? Many of the things you link to seem a bit disjointed to me because it is people experiencing different problems that aren't as related as you might think. More comments inline below.> On Oct 9, 2017, at 7:24 AM, Daniel Peebles via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > Hi all, > > I'm trying to understand why LLVM uses @rpath as the install name in its dynamic libraries on macOS. This complicates the process of linking against libLLVM from third-party tools and isn't nearly as common a practice on macOS as it is on Linux.I very much disagree. I think rpaths are very common on Darwin platforms. We see them all over the place.> I tried tracing through history on the LLVM repo and the main thing I could find was a commit from ages ago saying that the libraries are now relocatable. While that's indeed true, rpaths put a burden on all consumers of the library (who now need to pass special flags to their linker) and unless there's a pressing need for them, most libraries avoid the practice. Indeed, the use of @rpath in LLVM seems to have been a bit confusing and has undergone dozens of changes over time, with reverts, changes to and from @executable_path (a related but different relative path inside the macOS loader), and so on. > > Here's the commit that seemed to introduce it: https://github.com/llvm-mirror/llvm/commit/fb24ccfa32c12dc7ab12b70f2c3db404dbab7581 <https://github.com/llvm-mirror/llvm/commit/fb24ccfa32c12dc7ab12b70f2c3db404dbab7581> > Here's one of the reverts of the reverts: https://github.com/llvm-mirror/llvm/commit/fb24ccfa32c12dc7ab12b70f2c3db404dbab7581 <https://github.com/llvm-mirror/llvm/commit/fb24ccfa32c12dc7ab12b70f2c3db404dbab7581> > Here's the switch to @executable_path: https://github.com/llvm-mirror/llvm/commit/692c94c1c9b5642cecb8fc7b6fa5ce3145b70ff1 <https://github.com/llvm-mirror/llvm/commit/692c94c1c9b5642cecb8fc7b6fa5ce3145b70ff1> > Here it is switching back to @rpath: https://github.com/llvm-mirror/llvm/commit/37c04a0d4d0df0e2efddcc76a1036a9fc384e61a <https://github.com/llvm-mirror/llvm/commit/37c04a0d4d0df0e2efddcc76a1036a9fc384e61a>The chain of switching back and forth between @rpath and @executable_path pretty clearly illustrates that we *had* a problem with how we are setting the rpaths, not what the problem is. The reason I say *had* is because you left off one really important commit on the chain: https://github.com/llvm-mirror/llvm/commit/3a876bdaec4dd8e43d21e0ce369f17db274431c0 <https://github.com/llvm-mirror/llvm/commit/3a876bdaec4dd8e43d21e0ce369f17db274431c0> That patch I deleted all the code that sets the CMAKE_* global variables, and instead we set up rpaths on a per-library basis. I also moved to @loader_path, which is a more modern version of @executable_path.> > And here's an unanswered question about why it's there in the first place: http://lists.llvm.org/pipermail/llvm-dev/2016-January/094463.html <http://lists.llvm.org/pipermail/llvm-dev/2016-January/094463.html>I don't believe this comment is so much about setting the rpath, but rather the silly mechanism that we used in CMake to do it. I believe this has been settled since then.> Along with a downstream issue that needed to work around it: https://github.com/Homebrew/legacy-homebrew/issues/47149 <https://github.com/Homebrew/legacy-homebrew/issues/47149>The problem here is that libc++ is linking libcxxabi via @rpath instead of @loader_path. Also there are some historical differences between the way the Darwin build is done and the other build platforms. I believe that libcxx re-exporting libcxxabi is something we haven't historically done on Darwin, but I could be mis-remembering that.> And another downstream issue that outright patches out LLVM's @rpath support (authored by me): https://github.com/NixOS/nixpkgs/pull/30150 <https://github.com/NixOS/nixpkgs/pull/30150>This patch won't do anything against LLVM trunk and seems to be applying to LLVM 3.5. If you're looking for us to fix this on an old LLVM release, I don't expect you'll get much traction.> > Anyway, I'm curious what the use case is for a relocatable libLLVM.dylib at the cost of downstream confusion. If it is indeed essential, could it be made an option in the build so that those of us who don't need relocatable LLVM libraries don't need to hack at the build files to turn it off?libLLVM must be relocatable for many of our use cases. If it is a problem for you, we can certainly consider an option to disable that, although I'm more curious what the actual problem is. Creating a relocatable library shouldn't be causing downstream problems. -Chris> > Thanks, > Dan > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171009/bd65edb3/attachment.html>
Going a bit out of order, I think there are three things going on here: 1. My patch in nixpkgs: I think you're incorrect there; my patch isn't just for LLVM 3.5; you'll see that nixpkgs has packages for LLVM (3.4) 3.5, 3.7, 3.8, 3.9, 4, and 5, and I patch each of them slightly differently to disable rpath there. Each of them involves some on-the-fly patching of CMake files but they're in different places and work differently each time. 2. My motivation: we package a ton of FOSS libraries for macOS in nixpkgs and I recently made a change to avoid jumping through hoops to support rpaths, because the only library I've found in our packages that goes out of its way to enable rpaths (relocatable libraries are less relevant in the Nix world) is LLVM. The basic point is that using rpath in a library imposes a burden on any consumer of the library (to pass -rpath into the linker invocation of the final executable, which is probably more painful in nix than elsewhere because LLVM lives in its own folder rather than somewhere global like /usr/lib) 3. The background for rpath usage in LLVM: fully admit that I'm probably wrong here. I guess rpaths are much more common in graphical apps on macOS which I admittedly don't build very often, and it took me a long time to figure out the chain of commits and underlying motivations for using rpath as the explicit install name of the dylib, so I probably missed key steps there. Given #2, it seems like we'd be best served by just creating a CMake boolean flag for "use rpaths" or something like that. I'd set it to false and everyone else would use the current value for it. I'd be happy to submit such a patch if what I'm saying sounds reasonable. On Mon, Oct 9, 2017 at 1:48 PM, Chris Bieneman <beanz at apple.com> wrote:> Can you please describe the actual problem you're trying to fix? > > Many of the things you link to seem a bit disjointed to me because it is > people experiencing different problems that aren't as related as you might > think. > > More comments inline below. > > On Oct 9, 2017, at 7:24 AM, Daniel Peebles via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > > Hi all, > > I'm trying to understand why LLVM uses @rpath as the install name in its > dynamic libraries on macOS. This complicates the process of linking against > libLLVM from third-party tools and isn't nearly as common a practice on > macOS as it is on Linux. > > > I very much disagree. I think rpaths are very common on Darwin platforms. > We see them all over the place. > > I tried tracing through history on the LLVM repo and the main thing I > could find was a commit from ages ago saying that the libraries are now > relocatable. While that's indeed true, rpaths put a burden on all consumers > of the library (who now need to pass special flags to their linker) and > unless there's a pressing need for them, most libraries avoid the practice. > Indeed, the use of @rpath in LLVM seems to have been a bit confusing and > has undergone dozens of changes over time, with reverts, changes to and > from @executable_path (a related but different relative path inside the > macOS loader), and so on. > > Here's the commit that seemed to introduce it: https://github.com/llvm- > mirror/llvm/commit/fb24ccfa32c12dc7ab12b70f2c3db404dbab7581 > Here's one of the reverts of the reverts: https://github.com/ > llvm-mirror/llvm/commit/fb24ccfa32c12dc7ab12b70f2c3db404dbab7581 > Here's the switch to @executable_path: https:// > github.com/llvm-mirror/llvm/commit/692c94c1c9b5642cecb8fc7b6fa5ce > 3145b70ff1 > Here it is switching back to @rpath: https://github.com/ > llvm-mirror/llvm/commit/37c04a0d4d0df0e2efddcc76a1036a9fc384e61a > > > The chain of switching back and forth between @rpath and @executable_path > pretty clearly illustrates that we *had* a problem with how we are setting > the rpaths, not what the problem is. The reason I say *had* is because you > left off one really important commit on the chain: > > https://github.com/llvm-mirror/llvm/commit/3a876bdaec4dd8e43d21e0ce369f17 > db274431c0 > > That patch I deleted all the code that sets the CMAKE_* global variables, > and instead we set up rpaths on a per-library basis. I also moved to > @loader_path, which is a more modern version of @executable_path. > > > And here's an unanswered question about why it's there in the first place: > http://lists.llvm.org/pipermail/llvm-dev/2016-January/094463.html > > > I don't believe this comment is so much about setting the rpath, but > rather the silly mechanism that we used in CMake to do it. I believe this > has been settled since then. > > Along with a downstream issue that needed to work around it: > https://github.com/Homebrew/legacy-homebrew/issues/47149 > > > The problem here is that libc++ is linking libcxxabi via @rpath instead of > @loader_path. Also there are some historical differences between the way > the Darwin build is done and the other build platforms. I believe that > libcxx re-exporting libcxxabi is something we haven't historically done on > Darwin, but I could be mis-remembering that. > > And another downstream issue that outright patches out LLVM's @rpath > support (authored by me): https://github.com/NixOS/nixpkgs/pull/30150 > > > This patch won't do anything against LLVM trunk and seems to be applying > to LLVM 3.5. If you're looking for us to fix this on an old LLVM release, I > don't expect you'll get much traction. > > > Anyway, I'm curious what the use case is for a relocatable libLLVM.dylib > at the cost of downstream confusion. If it is indeed essential, could it be > made an option in the build so that those of us who don't need relocatable > LLVM libraries don't need to hack at the build files to turn it off? > > > libLLVM must be relocatable for many of our use cases. If it is a problem > for you, we can certainly consider an option to disable that, although I'm > more curious what the actual problem is. Creating a relocatable library > shouldn't be causing downstream problems. > > -Chris > > > Thanks, > Dan > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171009/cc00c4e7/attachment-0001.html>