llvm dev - Oct 2019 - contributing llvm-install-name-tool

Hey Michael,

I completely agree that setting the rpath properly the first time around is
much preferred.  But, changes to the binary, particularly during
development is much quicker.

Prebuilt libraries which are being repackaged is one use case that is
missed.

In the past, I’ve even used it to repair am incorrectly built library which
was missing the library name.

There are a few different approaches to implementing this, and I am not
particularly tied to one.  I think that selecting the one that is in the
best interest of the project and the most maintainable long term would be
best.

On Fri, Oct 11, 2019 at 10:12 PM Michael Trent <mtrent at apple.com>
wrote:
> Hi there,
>
> This tool has been around for a long time. Originally it gave people a way
> to move a dylib to a new location without rebuilding, and then as a way to
> make an existing dylib embeddable (@rpath). I wonder if this tool is really
> necessary. It’s always better to make the dylib with the correct install
> name at build time, including @rpath, etc.. Most dylibs we see any more are
> either installed in fixed file system locations by OS vendors (i.e., Apple)
> or are embedded @rpath dylibs installed by everyone else.
>
> Is there a use-case for install_name_tool that I have overlooked?
>
> Or Is that small set of dylibs meant to be installed in fixed file system
> locations large / complex enough that people really need to install them in
> many different locations without simply relinking the files?
>
> Turning to practical matters, is your intent to have the load-command
> rewriting code live in the tool? Or are you proposing adding support for
> building/swapping/rewriting Object files to libObject?
>
> MDT
>
>
> On Oct 11, 2019, at 10:31 AM, Alexander Shaposhnikov <
> alexander.v.shaposhnikov at gmail.com> wrote:
>
> Hey everyone!
> Recently there has been some progress on LLVM-based tools for manipulating
> MachO binaries: llvm-objcopy has been gaining a lot of important bits to
> support MachO (it's relatively close to the point where one can
implement
> the strip-like functionality), llvm-lipo is functional and supports most of
> cctools' lipo options
(https://llvm.org/docs/CommandGuide/llvm-lipo.html
> ).
> There is another useful utility called install_name_tool (see e.g.
> https://www.unix.com/man-page/osx/1/install_name_tool/ ),
> this tool is capable of changing the rpaths, the names of the dependent
> shared libraries, etc. The way it works -  install-name-tool (from cctools)
> generates the new list of MachO load commands and the new MachO header, the
> other parts of the binary are copied over (in particular, if I'm not
> mistaken, it assumes that the new "prefix" (MachO header + load
commands)
> fits into the binary so the offsets don't need to be recalculated,
> otherwise suggests to relink the binary with -headerpad /
> -headerpad_max_install_names / reports an error).  One possible (and
> simple) approach is to implement llvm-install-name-tool in a similar
> fashion: use libObject to parse the input binary and rebuild the list of
> load commands (analogously to what cctools' install_name_tool does (and
> with the same limitations)). I'd like to ask for your general feedback
/
> thoughts / suggestions on contributing llvm-install-name-tool to LLVM with
> the goal to be a drop-in replacement for cctools' install_name_tool.
> Thanks,
> Alex
>
>
> --
Saleem Abdulrasool
compnerd (at) compnerd (dot) org
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191014/6268326f/attachment.html>

"There are a few different approaches to implementing this, and I am not
particularly tied to one.  I think that selecting the one that is in the
best interest of the project and the most maintainable long term would be
best"

- I agree.

Regarding the implementation I can imagine different options here, but need
to take a closer look / play with the code a little bit to see how things
work. Hypothetical approaches: (A) build a standalone tool (similarly to
llvm-lipo) (B) extend llvm-objcopy & add a new "driver" (similarly
to how
llvm-strip is implemented on top of llvm-objcopy) (in particular,
llvm-install-name-tool would be a symbolic link to llvm-objcopy).  As I've
mentioned above - I need to take a closer look at the code before making a
decision / proposing a solution, there are different considerations on my
mind here, e.g. install-name-tool appears to be simpler than objcopy is,
and probably doesn't require the full power of the mutable model of an
object file (implemented inside llvm-objcopy).

Kind regards,
Alex

On Mon, Oct 14, 2019 at 10:21 AM Saleem Abdulrasool <compnerd at
compnerd.org>
wrote:
> Hey Michael,
>
> I completely agree that setting the rpath properly the first time around
> is much preferred.  But, changes to the binary, particularly during
> development is much quicker.
>
> Prebuilt libraries which are being repackaged is one use case that is
> missed.
>
> In the past, I’ve even used it to repair am incorrectly built library
> which was missing the library name.
>
> There are a few different approaches to implementing this, and I am not
> particularly tied to one.  I think that selecting the one that is in the
> best interest of the project and the most maintainable long term would be
> best.
>
> On Fri, Oct 11, 2019 at 10:12 PM Michael Trent <mtrent at apple.com>
wrote:
>
>> Hi there,
>>
>> This tool has been around for a long time. Originally it gave people a
>> way to move a dylib to a new location without rebuilding, and then as a
way
>> to make an existing dylib embeddable (@rpath). I wonder if this tool is
>> really necessary. It’s always better to make the dylib with the correct
>> install name at build time, including @rpath, etc.. Most dylibs we see
any
>> more are either installed in fixed file system locations by OS vendors
>> (i.e., Apple) or are embedded @rpath dylibs installed by everyone else.
>>
>> Is there a use-case for install_name_tool that I have overlooked?
>>
>> Or Is that small set of dylibs meant to be installed in fixed file
system
>> locations large / complex enough that people really need to install
them in
>> many different locations without simply relinking the files?
>>
>> Turning to practical matters, is your intent to have the load-command
>> rewriting code live in the tool? Or are you proposing adding support
for
>> building/swapping/rewriting Object files to libObject?
>>
>> MDT
>>
>>
>> On Oct 11, 2019, at 10:31 AM, Alexander Shaposhnikov <
>> alexander.v.shaposhnikov at gmail.com> wrote:
>>
>> Hey everyone!
>> Recently there has been some progress on LLVM-based tools for
>> manipulating MachO binaries: llvm-objcopy has been gaining a lot of
>> important bits to support MachO (it's relatively close to the point
where
>> one can implement the strip-like functionality), llvm-lipo is
functional
>> and supports most of cctools' lipo options (
>> https://llvm.org/docs/CommandGuide/llvm-lipo.html).
>> There is another useful utility called install_name_tool (see e.g.
>> https://www.unix.com/man-page/osx/1/install_name_tool/ ),
>> this tool is capable of changing the rpaths, the names of the dependent
>> shared libraries, etc. The way it works -  install-name-tool (from
cctools)
>> generates the new list of MachO load commands and the new MachO header,
the
>> other parts of the binary are copied over (in particular, if I'm
not
>> mistaken, it assumes that the new "prefix" (MachO header +
load commands)
>> fits into the binary so the offsets don't need to be recalculated,
>> otherwise suggests to relink the binary with -headerpad /
>> -headerpad_max_install_names / reports an error).  One possible (and
>> simple) approach is to implement llvm-install-name-tool in a similar
>> fashion: use libObject to parse the input binary and rebuild the list
of
>> load commands (analogously to what cctools' install_name_tool does
(and
>> with the same limitations)). I'd like to ask for your general
feedback /
>> thoughts / suggestions on contributing llvm-install-name-tool to LLVM
with
>> the goal to be a drop-in replacement for cctools'
install_name_tool.
>> Thanks,
>> Alex
>>
>>
>> --
> Saleem Abdulrasool
> compnerd (at) compnerd (dot) org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191014/eaf15aab/attachment.html>

The install_name_tool command is one of many tools that simply rewrite load
commands. They are all exactly the same except for the non-trivial matter of
user interface:

1) read a file,
2) rebuild some or all of the load-command buffer,
3) write out a file.

They all have the same limitations in that the location of the segments in a
non-relocatable image cannot be moved.

Ideally all of the logic for doing this operation is somewhere central that’s
easy to maintain and easy to reuse. Then people could work out the user
interface problem of “how to specify what the tool should do” in some way that
makes sense. While a tool like install_name_tool isn’t that interesting to me, a
tool (or a toolkit, library, framework, ...) that rewrites all load commands is
a lot more interesting.

MDT

> On Oct 14, 2019, at 10:34 AM, Alexander Shaposhnikov
<alexander.v.shaposhnikov at gmail.com> wrote:
> 
> "There are a few different approaches to implementing this, and I am
not particularly tied to one.  I think that selecting the one that is in the
best interest of the project and the most maintainable long term would be
best"
> 
> - I agree.  
> 
> Regarding the implementation I can imagine different options here, but need
to take a closer look / play with the code a little bit to see how things work.
Hypothetical approaches: (A) build a standalone tool (similarly to llvm-lipo)
(B) extend llvm-objcopy & add a new "driver" (similarly to how
llvm-strip is implemented on top of llvm-objcopy) (in particular,
llvm-install-name-tool would be a symbolic link to llvm-objcopy).  As I've
mentioned above - I need to take a closer look at the code before making a
decision / proposing a solution, there are different considerations on my mind
here, e.g. install-name-tool appears to be simpler than objcopy is, and probably
doesn't require the full power of the mutable model of an object file
(implemented inside llvm-objcopy).
> 
> Kind regards,
> Alex
> 
> On Mon, Oct 14, 2019 at 10:21 AM Saleem Abdulrasool <compnerd at
compnerd.org <mailto:compnerd at compnerd.org>> wrote:
> Hey Michael,
> 
> I completely agree that setting the rpath properly the first time around is
much preferred.  But, changes to the binary, particularly during development is
much quicker.
> 
> Prebuilt libraries which are being repackaged is one use case that is
missed.
> 
> In the past, I’ve even used it to repair am incorrectly built library which
was missing the library name.
> 
> There are a few different approaches to implementing this, and I am not
particularly tied to one.  I think that selecting the one that is in the best
interest of the project and the most maintainable long term would be best.
> 
> On Fri, Oct 11, 2019 at 10:12 PM Michael Trent <mtrent at apple.com
<mailto:mtrent at apple.com>> wrote:
> Hi there,
> 
> This tool has been around for a long time. Originally it gave people a way
to move a dylib to a new location without rebuilding, and then as a way to make
an existing dylib embeddable (@rpath). I wonder if this tool is really
necessary. It’s always better to make the dylib with the correct install name at
build time, including @rpath, etc.. Most dylibs we see any more are either
installed in fixed file system locations by OS vendors (i.e., Apple) or are
embedded @rpath dylibs installed by everyone else.
> 
> Is there a use-case for install_name_tool that I have overlooked?
> 
> Or Is that small set of dylibs meant to be installed in fixed file system
locations large / complex enough that people really need to install them in many
different locations without simply relinking the files?
> 
> Turning to practical matters, is your intent to have the load-command
rewriting code live in the tool? Or are you proposing adding support for
building/swapping/rewriting Object files to libObject?
> 
> MDT
> 
> 
>> On Oct 11, 2019, at 10:31 AM, Alexander Shaposhnikov
<alexander.v.shaposhnikov at gmail.com <mailto:alexander.v.shaposhnikov at
gmail.com>> wrote:
>> 
>> Hey everyone!  
>> Recently there has been some progress on LLVM-based tools for
manipulating MachO binaries: llvm-objcopy has been gaining a lot of important
bits to support MachO (it's relatively close to the point where one can
implement the strip-like functionality), llvm-lipo is functional and supports
most of cctools' lipo options
(https://llvm.org/docs/CommandGuide/llvm-lipo.html
<https://llvm.org/docs/CommandGuide/llvm-lipo.html>).
>> There is another useful utility called install_name_tool (see e.g.
https://www.unix.com/man-page/osx/1/install_name_tool/
<https://www.unix.com/man-page/osx/1/install_name_tool/> ),
>> this tool is capable of changing the rpaths, the names of the dependent
shared libraries, etc. The way it works -  install-name-tool (from cctools)
generates the new list of MachO load commands and the new MachO header, the
other parts of the binary are copied over (in particular, if I'm not
mistaken, it assumes that the new "prefix" (MachO header + load
commands) fits into the binary so the offsets don't need to be recalculated,
otherwise suggests to relink the binary with -headerpad /
-headerpad_max_install_names / reports an error).  One possible (and simple)
approach is to implement llvm-install-name-tool in a similar fashion: use
libObject to parse the input binary and rebuild the list of load commands
(analogously to what cctools' install_name_tool does (and with the same
limitations)). I'd like to ask for your general feedback / thoughts /
suggestions on contributing llvm-install-name-tool to LLVM with the goal to be a
drop-in replacement for cctools' install_name_tool.
>> Thanks,
>> Alex
> 
> -- 
> Saleem Abdulrasool
> compnerd (at) compnerd (dot) org
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191014/a213811a/attachment.html>