llvm dev - Dec 2011 - [LLVMdev] [RFC]Extending lib/Linker to support bitcode "shared objects"

On Tue, Dec 13, 2011 at 9:27 PM, Rafael Ávila de Espíndola <
rafael.espindola at gmail.com> wrote:
>
> >     $ llc bar.bc -filetype=obj -o bar.o
> >     $ clang -shared -o bar.so bar.o
> >     $ clang -use-gold-plugin foo.o bar.so -o t
> >
> >     Is that correct? In particular, "lld t" should show a
dependency on
> bar.
> >     Any particular reason for not adding this to the plugin api?
> >
> > The result is a native .so here. My goal is to have a bitcode result.
> >
> > gold plugin with mods supports that as well, but it would be nice to
> > avoid dependency on gold and gold plugin to simplify things.
>
> Sorry, I meant equivalent only in the produced executable (t). In the
> above example the bar.bc is your "shared IL library" which I
assume
> already exists. You want to produce a executable (t) which has a
> dependency on a shared library with the symbols defined in foo.bc.
>
> Is that the use case?
>
Almost. The difference is that I want to have a bitcode module as the
result of linking, not executable file.
>
> > I believe that Emscripten does not use gold and gold plugin at the
> > moment. They use
> > llvm-link:
https://github.com/kripken/emscripten/wiki/Building-Projects
>
> I think that is the current implementation, yes.
>
> > I fully agree that all the features I want from the bitcode linker may
> > be achieved with gold + gold plugin (with mods to both parts), but I
> > would like to stay away from this dependency. Partly, because
> > maintaining modified gold and gold plugin is no fun, partly because
gold
> > + gold plugin is hardly broken under cygwin.
>
> I understand. I was just pointing that emacscriten case looks different
> from what you have. I don't see where an LLVM IL library (libc in your
> example) would fit. All that is needed is a stub with the functions that
> will be defined in JS in the end.
>
bitcode "so" is supposed to have the form of:

define void @printf() {
  ret void
}
...

In this case, Empscripten might want to generate such stubs based on the
list of javascript functions they provide and in my case, the generation of
this LL stub is a simple script based on nm, sed and llvm-as that takes
native .so as the input and produces a bitcode file with stub definitions
of the public symbols.

This is about the same use case as Emscripten with the only difference in
the source of the defined symbols (native .so vs js file)

Is it more clear now? If not, I would like to give it another try and write
much more details and examples.

krasin
>
> Cheers,
> Rafael
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20111213/3a3af8c9/attachment.html>

> Is it more clear now? If not, I would like to give it another try and
> write much more details and examples.
I am still not completely sure I understand the use case. In particular
since you say you want a bitcode module in the end, I don't understand
how this is different from using llvm-link, it doesn't complain about
undefined symbols...

Can you provide an example? What are your inputs. Are they IL or ELF?
What are the outputs that you want? Are they IL or ELF?
> krasin
Cheers,
Rafael

On Wed, Dec 14, 2011 at 10:38 AM, Rafael Ávila de Espíndola <
rafael.espindola at gmail.com> wrote:
> > Is it more clear now? If not, I would like to give it another try and
> > write much more details and examples.
>
> I am still not completely sure I understand the use case. In particular
> since you say you want a bitcode module in the end, I don't understand
> how this is different from using llvm-link, it doesn't complain about
> undefined symbols...
>
Complaining about undefined symbols is the next step. It's already
implemented in my branch:
https://github.com/krasin/bitlink/commit/be222a2863a989666d4925e5344d0c84cac8e06b

>
> Can you provide an example? What are your inputs. Are they IL or ELF?
> What are the outputs that you want? Are they IL or ELF?
>
All the inputs and all the outputs are IL. Some IL inputs are just stubs
which have only functions with empty bodies.
The output IL file is supposed to have all the symbols resolved, statically
or dynamically (i.e. there's a stub that has that symbols defined).
In case of Emscripten, that's enough.
In our case (PNaCl), we also add metadata to the output IL file, so that it
would be possible to track which shared libraries are actually needed.

As for "real life" example, we may want to consider a developer on
Windows
machine who wants to write a portable program (that will run inside NaCl)
using libpng and glibc using PNaCl toolchain that would provide portable
bitcode stubs for glibc and libpng port from naclports repositories (with
link-time portable stubs as well). In this case, the developer wants to
create a bitcode file that will be treated by PNaCl as executable (and
would be translated to the target architecture before the actual run),
linked dynamically with glibc and libpng. He may achieve this by using
modified gold + modified gold plugin from the PNaCl toolchain (but Windows
would prevent him from this path, see cygwin notes in my previous
messages), or he can use the bitcode linker, I'm currently working on. In
this case, the developer will know if all the symbols are resolved and will
not have a dependency on gold linker.



>
> > krasin
>
> Cheers,
> Rafael
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20111215/06e8108d/attachment.html>