Daniel Liew
2013-Dec-20 10:39 UTC
[LLVMdev] [LLVM] What has happened to LLVM bitcode archive support?
Hi Rafael and other LLVM devs, I'm currently upgrading a project that uses LLVM that links a bitcode archive (a C library) with a module. Originally we used Linker::LinkInFile() but that was removed by r172749. So I started looking for an alternative and I found Archive::findModulesDefiningSymbols() which looked very promising as it would allow me to very easily implement linking a bitcode archive with a module efficiently (i.e. only call Linker::LinkModules() on the relevant bitcode modules inside the bitcode archive). Before I started I thought I'd better check that this API was still going to be available in the upcoming 3.4 release and to my dismay ``llvm/Bitcode/Archive.h`` was moved by you in r184083 and finally removed by you in r186197. So... * Is LLVM completely moving away from the idea have bitcode archive runtime libraries? The LLVM Make build system seems to support building these but LLVM doesn't seem to have an API for linking runtime bitcode archives into a bitcode module any more (or have I missed something?). I have seen suggestions of using the LLVM gold plug-in but that's not an API I can use, is it? * I'm guessing I'm supposed to use the Object/Archive.h interface now? This doesn't seem provide a nice interface for getting the set of Modules I want to link with, so what am I supposed to do instead? Any help would be appreciated, Thanks, Dan Liew.
Rafael Espíndola
2013-Dec-20 14:30 UTC
[LLVMdev] [LLVM] What has happened to LLVM bitcode archive support?
On 20 December 2013 05:39, Daniel Liew <daniel.liew at imperial.ac.uk> wrote:> Hi Rafael and other LLVM devs, > > I'm currently upgrading a project that uses LLVM that links a bitcode > archive (a C library) with a module. Originally we used > Linker::LinkInFile() but that was removed by r172749. So I started > looking for an alternative and I found > Archive::findModulesDefiningSymbols() which looked very promising as > it would allow me to very easily implement linking a bitcode archive > with a module efficiently (i.e. only call Linker::LinkModules() on the > relevant bitcode modules inside the bitcode archive). > > Before I started I thought I'd better check that this API was still > going to be available in the upcoming 3.4 release and to my dismay > ``llvm/Bitcode/Archive.h`` was moved by you in r184083 and finally > removed by you in r186197. > > So... > > * Is LLVM completely moving away from the idea have bitcode archive > runtime libraries? The LLVM Make build system seems to support > building these but LLVM doesn't seem to have an API for linking > runtime bitcode archives into a bitcode module any more (or have I > missed something?). I have seen suggestions of using the LLVM gold > plug-in but that's not an API I can use, is it? > > * I'm guessing I'm supposed to use the Object/Archive.h interface now? > This doesn't seem provide a nice interface for getting the set of > Modules I want to link with, so what am I supposed to do instead?We are going this way, yes. The overview is * In the old days llvm had its own archive index format for IL files and special code to use it. * llvm-ld was very incomplete, and not designed to grow into real linker, so an alternative for LTO was needed. * apple implemented libLTO and changed their linker to use it. A wrapper (LLVMgold.so) was created for gold and bfd. * Using this the gnu nm, ar and ld (and gold) were able to do LTO. In particular, a IL files can be put in archives and show up in the regular symbol table. * llvm-ld was deprecated and then removed. * with nothing able to read the old LLVM only symbol table, that was also removed along with other archive related now dead code. The more recent story is that llvm has grow support for object files, so that llvm-ar is now a "real" (if immature) ar and lld is growing to be a real linker. Currently llvm-ar is missing support for putting IL files in the symbol table. The main reason is the desire to now get this right this time, which required quiet a bit of refactoring. I posted what I think is the last big patch for review (making the Mangler not depend on target). With that in things should move fairly quickly to have llvm-ar just work with IL files and with that lib/Object should have the features you need.> Any help would be appreciated, > > Thanks, > Dan Liew.Cheers, Rafael
Iain Sandoe
2013-Dec-20 16:03 UTC
[LLVMdev] [LLVM] What has happened to LLVM bitcode archive support?
Hi Rafael, On 20 Dec 2013, at 14:30, Rafael Espíndola wrote:> On 20 December 2013 05:39, Daniel Liew <daniel.liew at imperial.ac.uk> wrote: >> Hi Rafael and other LLVM devs, >> >> I'm currently upgrading a project that uses LLVM that links a bitcode >> archive (a C library) with a module. Originally we used >> Linker::LinkInFile() but that was removed by r172749. So I started >> looking for an alternative and I found >> Archive::findModulesDefiningSymbols() which looked very promising as >> it would allow me to very easily implement linking a bitcode archive >> with a module efficiently (i.e. only call Linker::LinkModules() on the >> relevant bitcode modules inside the bitcode archive). >> >> Before I started I thought I'd better check that this API was still >> going to be available in the upcoming 3.4 release and to my dismay >> ``llvm/Bitcode/Archive.h`` was moved by you in r184083 and finally >> removed by you in r186197. >> >> So... >> >> * Is LLVM completely moving away from the idea have bitcode archive >> runtime libraries? The LLVM Make build system seems to support >> building these but LLVM doesn't seem to have an API for linking >> runtime bitcode archives into a bitcode module any more (or have I >> missed something?). I have seen suggestions of using the LLVM gold >> plug-in but that's not an API I can use, is it? >> >> * I'm guessing I'm supposed to use the Object/Archive.h interface now? >> This doesn't seem provide a nice interface for getting the set of >> Modules I want to link with, so what am I supposed to do instead? > > We are going this way, yes. The overview is > > * In the old days llvm had its own archive index format for IL files > and special code to use it. > * llvm-ld was very incomplete, and not designed to grow into real > linker, so an alternative for LTO was needed. > * apple implemented libLTO and changed their linker to use it. A > wrapper (LLVMgold.so) was created for gold and bfd. > * Using this the gnu nm, ar and ld (and gold) were able to do LTO. In > particular, a IL files can be put in archives and show up in the > regular symbol table. > * llvm-ld was deprecated and then removed. > * with nothing able to read the old LLVM only symbol table, that was > also removed along with other archive related now dead code. > > The more recent story is that llvm has grow support for object files, > so that llvm-ar is now a "real" (if immature) ar and lld is growing to > be a real linker. Currently llvm-ar is missing support for putting IL > files in the symbol table. The main reason is the desire to now get > this right this time, which required quiet a bit of refactoring. I > posted what I think is the last big patch for review (making the > Mangler not depend on target). With that in things should move fairly > quickly to have llvm-ar just work with IL files and with that > lib/Object should have the features you need.Could I just note (for the record) there is/are [at least one ;)] bare-metal targets that have neither static nor dynamic linkers. We have one such (out-of-tree) target and make use of llvm-link to link bitcode - the linked bit code is translated directly into the executable. We would, however, welcome a solution that allowed the 'link' to be extended to include bitcode archives. However, it's probably a non-starter (project constraints) to implement a static linker solely for this purpose. cheers Iain
Maybe Matching Threads
- [LLVMdev] [LLVM] What has happened to LLVM bitcode archive support?
- [LLVMdev] [LLVM] What has happened to LLVM bitcode archive support?
- [LLVMdev] How to do bitcode archive linking correctly?
- [LLVMdev] How to do bitcode archive linking correctly?
- [LLVMdev] [LLVM] What has happened to LLVM bitcode archive support?