thr3ads.net - llvm dev - [LLVMdev] Shared library building problems on Darwin [Jan 2005]

If this information is useful, please help other people find it:
Share via:

Michael McCracken

2005-Jan-07 02:39 UTC

[LLVMdev] Shared library building problems on Darwin

Hi, a while back I wrote that the llvm makefiles didn't create the
correct kind of file for use on darwin with -load.

Since then, both the shared library and makefile system have been
overhauled significantly.
So I checked again - as updated from CVS, the current makefiles don't
build the right object type on darwin.

If you follow the advice of 'Writing an LLVM Pass" tutorial, you will
create (say) llvm/lib/Transforms/PMFBuild/ and fill it with the
appropriate files.

Then, running 'make' in llvm/lib/Transforms/PMFBuild generates a bunch
of files in llvm/Debug/lib/:
% cd llvm/Debug/lib
% ll *PMF*
-rw-r--r--    1 mike     staff      385584 Jan  6 18:16 PMFBuild.o
-rwxr-xr-x    1 mike     staff     1053828 Jan  6 18:16 libPMFBuild.0.0.0.dylib
lrwxr-xr-x    1 mike     staff          23 Jan  6 18:16
libPMFBuild.0.dylib -> libPMFBuild.0.0.0.dylib
-rw-r--r--    1 mike     staff      385600 Jan  6 18:16 libPMFBuild.a
lrwxr-xr-x    1 mike     staff          23 Jan  6 18:16
libPMFBuild.dylib -> libPMFBuild.0.0.0.dylib
-rwxr-xr-x    1 mike     staff         921 Jan  6 18:16 libPMFBuild.la

I think this means I'm running in OBJDIR==SRCDIR mode, but you tell me. 
I'm not up to speed on the new makefile system.

Note the file types:
% file *PMF*           
PMFBuild.o:              Mach-O object ppc
libPMFBuild.0.0.0.dylib: Mach-O dynamically linked shared library ppc
libPMFBuild.0.dylib:     symbolic link to `libPMFBuild.0.0.0.dylib'
libPMFBuild.a:           current ar archive
libPMFBuild.dylib:       symbolic link to `libPMFBuild.0.0.0.dylib'
libPMFBuild.la:          ASCII English text

This may look OK, but we see that -load doesn't complain, but fails to
actually load it.
(Take my word that -pmfbuild would work if not for this, it does later.)

So I tried the same hack I found earlier:

--- Makefile.rules      3 Jan 2005 17:42:57 -0000       1.287
+++ Makefile.rules      7 Jan 2005 02:29:22 -0000
@@ -326,7 +326,7 @@
 LTCompile.CXX = $(LIBTOOL) $(LibTool.Flags) --mode=compile $(Compile.CXX)
 BCCompile.CXX = $(LLVMGXXWITHPATH) $(CPP.Flags) $(CompileCommonOpts) \
                 $(CXX.Flags) -c
-Link          = $(LIBTOOL) $(LibTool.Flags) --mode=link $(CXX) $(CPP.Flags) \
+Link          = $(LIBTOOL) $(LibTool.Flags) --mode=link $(CXX)
-module $(CPP.Flags) \
                $(CompileCommonOpts) $(LD.Flags) $(Strip)
 Relink        = $(LIBTOOL) $(LibTool.Flags) --mode=link $(CXX) $(CPP.Flags) \
                 $(CompileCommonOpts)

Removing the old *PMF* libraries and objects in llvm/Debug/lib/ and
re-running make in lib/Transforms/PMFBuild/ gives the following list
of files:

% ll *PMF*
-rw-r--r--    1 mike     staff      385584 Jan  6 18:31 PMFBuild.o
-rwxr-xr-x    1 mike     staff     1052968 Jan  6 18:31 libPMFBuild.0.0.0.so
lrwxr-xr-x    1 mike     staff          20 Jan  6 18:31
libPMFBuild.0.so -> libPMFBuild.0.0.0.so
-rw-r--r--    1 mike     staff      385600 Jan  6 18:31 libPMFBuild.a
-rwxr-xr-x    1 mike     staff         910 Jan  6 18:31 libPMFBuild.la
lrwxr-xr-x    1 mike     staff          20 Jan  6 18:31 libPMFBuild.so
-> libPMFBuild.0.0.0.so

% file *PMF*
PMFBuild.o:           Mach-O object ppc
libPMFBuild.0.0.0.so: Mach-O bundle ppc
libPMFBuild.0.so:     symbolic link to `libPMFBuild.0.0.0.so'
libPMFBuild.a:        current ar archive
libPMFBuild.la:       ASCII English text
libPMFBuild.so:       symbolic link to `libPMFBuild.0.0.0.so'

Note that the file is now a Mach-O bundle, not a Dynamically Linked
Library, and
it has the appropriate .so extension (thanks to libtool - Apple
actually seems to prefer to build bundles with no extension)

This is important, because on darwin, dynamically linked libraries and
'loadable modules' are not the same thing, as they are in object file
formats in use elsewhere. In order to be loaded dynamically after the
program is already running, the file has to be a "Mach-O Bundle".

So with these file types, it works:
/Users/mike/Documents/hpcl/LLVM/cfe-src/install/bin/gcc -g
tests/daxpy/daxpy.c -o tests/daxpy/daxpy
/Users/mike/Documents/hpcl/LLVM/llvm-cvs/Debug/bin//opt -f -stats
-debug-pass=Structure  -load
/Users/mike/Documents/hpcl/LLVM/llvm-cvs/Debug/lib//libPMFBuild.so
-buildpmf  -o tests/daxpy/daxpy-opt tests/daxpy/daxpy.bc
Pass Arguments:  -buildpmf -verify
Target Data Layout
Module Pass Manager
  Function Pass Manager
    Build PMF Structures
--  Build PMF Structures
    Immediate Dominators Construction
    Dominator Set Construction
--  Immediate Dominators Construction
    Module Verifier
--  Dominator Set Construction
--  Module Verifier
  Bytecode Writer
--Bytecode Writer

I'm not sure how best to fit this fix into the LLVM makefiles, but it
seems like there should be a way to toss in that -module option for
linking shared libraries only on Darwin. However, like I said, I'm not
up on the new system, and I just found the Link command and tried my
old hack.

Hope this helps,
-mike

-- 
Michael McCracken
UCSD CSE PhD Student
San Diego Supercomputer Center
http://www.cse.ucsd.edu/~mmccrack/

Reid Spencer

2005-Jan-10 23:25 UTC

head link

[LLVMdev] Shared library building problems on Darwin

Michael,

Chris and I have been having a discussion about this topic on the side.
I won't bore you with the details, but it boils down to this:

1. libtool (mklib) makes a distinction between "shared library" and
   "loadable module". 
2. On most systems shared library and loadable module are the same
   thing. That is, the dynamic linker uses dlopen/dlsym on a shared
   object just as a user program might.
3. The above isn't true for Darwin
4. Our makefiles insist on making "shared library" not loadable
modules
   when you specify SHARED_LIBRARY=1 in a makefile.

In most Unixes it doesn't matter but Darwin is "special" so it
matters
and you end up with the scenario you're seeing. The solution is to
permit both kinds of links to be done (one with -module and one
without). So, I'm going to add support for LOADABLE_MODULE=1 setting to
Makefile.rules so we can build both types of libraries. This should
clear up the problem on Darwin.

Unfortunately, my time is constrained right now. I'll attend to this as
soon as I'm able.

Thanks for your patience.

Reid.


  
On Thu, 2005-01-06 at 18:39, Michael McCracken wrote:> Hi, a while back I wrote that the llvm makefiles didn't create the
> correct kind of file for use on darwin with -load.
> 
> Since then, both the shared library and makefile system have been
> overhauled significantly.
> So I checked again - as updated from CVS, the current makefiles don't
> build the right object type on darwin.
> 
> If you follow the advice of 'Writing an LLVM Pass" tutorial, you
will
> create (say) llvm/lib/Transforms/PMFBuild/ and fill it with the
> appropriate files.
> 
> Then, running 'make' in llvm/lib/Transforms/PMFBuild generates a
bunch
> of files in llvm/Debug/lib/:
> % cd llvm/Debug/lib
> % ll *PMF*
> -rw-r--r--    1 mike     staff      385584 Jan  6 18:16 PMFBuild.o
> -rwxr-xr-x    1 mike     staff     1053828 Jan  6 18:16
libPMFBuild.0.0.0.dylib
> lrwxr-xr-x    1 mike     staff          23 Jan  6 18:16
> libPMFBuild.0.dylib -> libPMFBuild.0.0.0.dylib
> -rw-r--r--    1 mike     staff      385600 Jan  6 18:16 libPMFBuild.a
> lrwxr-xr-x    1 mike     staff          23 Jan  6 18:16
> libPMFBuild.dylib -> libPMFBuild.0.0.0.dylib
> -rwxr-xr-x    1 mike     staff         921 Jan  6 18:16 libPMFBuild.la
> 
> I think this means I'm running in OBJDIR==SRCDIR mode, but you tell me.
> I'm not up to speed on the new makefile system.
> 
> Note the file types:
> % file *PMF*           
> PMFBuild.o:              Mach-O object ppc
> libPMFBuild.0.0.0.dylib: Mach-O dynamically linked shared library ppc
> libPMFBuild.0.dylib:     symbolic link to `libPMFBuild.0.0.0.dylib'
> libPMFBuild.a:           current ar archive
> libPMFBuild.dylib:       symbolic link to `libPMFBuild.0.0.0.dylib'
> libPMFBuild.la:          ASCII English text
> 
> This may look OK, but we see that -load doesn't complain, but fails to
> actually load it.
> (Take my word that -pmfbuild would work if not for this, it does later.)
> 
> So I tried the same hack I found earlier:
> 
> --- Makefile.rules      3 Jan 2005 17:42:57 -0000       1.287
> +++ Makefile.rules      7 Jan 2005 02:29:22 -0000
> @@ -326,7 +326,7 @@
>  LTCompile.CXX = $(LIBTOOL) $(LibTool.Flags) --mode=compile $(Compile.CXX)
>  BCCompile.CXX = $(LLVMGXXWITHPATH) $(CPP.Flags) $(CompileCommonOpts) \
>                  $(CXX.Flags) -c
> -Link          = $(LIBTOOL) $(LibTool.Flags) --mode=link $(CXX)
$(CPP.Flags) \
> +Link          = $(LIBTOOL) $(LibTool.Flags) --mode=link $(CXX)
> -module $(CPP.Flags) \
>                 $(CompileCommonOpts) $(LD.Flags) $(Strip)
>  Relink        = $(LIBTOOL) $(LibTool.Flags) --mode=link $(CXX)
$(CPP.Flags) \
>                  $(CompileCommonOpts)
> 
> Removing the old *PMF* libraries and objects in llvm/Debug/lib/ and
> re-running make in lib/Transforms/PMFBuild/ gives the following list
> of files:
> 
> % ll *PMF*
> -rw-r--r--    1 mike     staff      385584 Jan  6 18:31 PMFBuild.o
> -rwxr-xr-x    1 mike     staff     1052968 Jan  6 18:31
libPMFBuild.0.0.0.so
> lrwxr-xr-x    1 mike     staff          20 Jan  6 18:31
> libPMFBuild.0.so -> libPMFBuild.0.0.0.so
> -rw-r--r--    1 mike     staff      385600 Jan  6 18:31 libPMFBuild.a
> -rwxr-xr-x    1 mike     staff         910 Jan  6 18:31 libPMFBuild.la
> lrwxr-xr-x    1 mike     staff          20 Jan  6 18:31 libPMFBuild.so
> -> libPMFBuild.0.0.0.so
> 
> % file *PMF*
> PMFBuild.o:           Mach-O object ppc
> libPMFBuild.0.0.0.so: Mach-O bundle ppc
> libPMFBuild.0.so:     symbolic link to `libPMFBuild.0.0.0.so'
> libPMFBuild.a:        current ar archive
> libPMFBuild.la:       ASCII English text
> libPMFBuild.so:       symbolic link to `libPMFBuild.0.0.0.so'
> 
> Note that the file is now a Mach-O bundle, not a Dynamically Linked
> Library, and
> it has the appropriate .so extension (thanks to libtool - Apple
> actually seems to prefer to build bundles with no extension)
> 
> This is important, because on darwin, dynamically linked libraries and
> 'loadable modules' are not the same thing, as they are in object
file
> formats in use elsewhere. In order to be loaded dynamically after the
> program is already running, the file has to be a "Mach-O Bundle".
> 
> So with these file types, it works:
> /Users/mike/Documents/hpcl/LLVM/cfe-src/install/bin/gcc -g
> tests/daxpy/daxpy.c -o tests/daxpy/daxpy
> /Users/mike/Documents/hpcl/LLVM/llvm-cvs/Debug/bin//opt -f -stats
> -debug-pass=Structure  -load
> /Users/mike/Documents/hpcl/LLVM/llvm-cvs/Debug/lib//libPMFBuild.so
> -buildpmf  -o tests/daxpy/daxpy-opt tests/daxpy/daxpy.bc
> Pass Arguments:  -buildpmf -verify
> Target Data Layout
> Module Pass Manager
>   Function Pass Manager
>     Build PMF Structures
> --  Build PMF Structures
>     Immediate Dominators Construction
>     Dominator Set Construction
> --  Immediate Dominators Construction
>     Module Verifier
> --  Dominator Set Construction
> --  Module Verifier
>   Bytecode Writer
> --Bytecode Writer
> 
> I'm not sure how best to fit this fix into the LLVM makefiles, but it
> seems like there should be a way to toss in that -module option for
> linking shared libraries only on Darwin. However, like I said, I'm not
> up on the new system, and I just found the Link command and tried my
> old hack.
> 
> Hope this helps,
> -mike-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20050110/64aff593/attachment.sig>

Reasonably Related Threads

Search for more reasonably related threads

llvm dev - Jan 2005 - [LLVMdev] Shared library building problems on Darwin

[LLVMdev] Shared library building problems on Darwin

[LLVMdev] Shared library building problems on Darwin

Reasonably Related Threads