On Sun, 2003-11-16 at 10:44, Chris Lattner wrote:> > In other words, I'd like to take a set of bytecode files, optimize them > > together even though they don't form a complete program, and then write > > out the new (optimized) bytecode files. It would be preferable to write > > them out to a single archive rather than to individual bytecode files > > again. Can this be done in LLVM today? If not, what would it take to > > implement? > > I'm not sure exactly what you want, but we do already have a form of this. > If you use the low-level llvm-link program (or gccld --link-as-library), > it will take all of the .bc files specified, link them together into a > _single_ bytecode file, and write that out (gccld performs IPO on the > result).Okay.> > Is this what you mean? Is there any reason to keep the .bc files distinct > like a ".a" file, or is it ok for your purposes to link them together into > a single unit, like a ".so" file?Well, the answer depends on whether the individual modules are separately loadable or not. Suppose the resulting bytecode file gets JIT loaded into a running program. Is it all or nothing? What if the calling program new it needed a specific module from the larger optimized "package"? Can the original modules, now combined into the one .bc file, get loaded individually? For example, using Java, suppose I wanted to use java.util.zip.CRC32 to compute a checksum in a program. That class (module) is delivered to me as a java.util.zip "package". That is, it is the result of running llvm-link to produce a single bytecode file that includes all the classes with java.util.zip prefix. But (for some mysterious reason) the java.util.zip/CRC32 class was not compiled/linked/optimized into my program; its just referenced from it. At runtime, the JIT needs to locate this module and compile/link it into my program. Now, if the CRC32 class (module) is part of a java.util.zip package (i.e. the output of llvm-link), what happens? Is the entire package loaded providing more than my program needs? Is the runtime linker smart enough to locate the CRC32 module in the optimized package output? Is it smart enough to load only the one module needed? Does the notion of module inside a module even exist? P.S. Bug 114 (-Wold-style-cast) has a patch ready. Reid -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20031116/a27d4af1/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20031116/a27d4af1/attachment.sig>
> > Is this what you mean? Is there any reason to keep the .bc files distinct > > like a ".a" file, or is it ok for your purposes to link them together into > > a single unit, like a ".so" file? > > Well, the answer depends on whether the individual modules are > separately loadable or not. Suppose the resulting bytecode file gets JIT > loaded into a running program. Is it all or nothing? What if the calling > program new it needed a specific module from the larger optimized > "package"? Can the original modules, now combined into the one .bc file, > get loaded individually?No, it's all or nothing. Once linked, they cannot be seperated (easily). However, especially when using the JIT, there is little overhead for running a gigantic program that only has 1% of the functions in it ever executed...> For example, using Java, suppose I wanted to use java.util.zip.CRC32 to > compute a checksum in a program. That class (module) is delivered to me > as a java.util.zip "package". That is, it is the result of running > llvm-link to produce a single bytecode file that includes all the > classes with java.util.zip prefix. But (for some mysterious reason) the > java.util.zip/CRC32 class was not compiled/linked/optimized into my > program; its just referenced from it. At runtime, the JIT needs to > locate this module and compile/link it into my program. Now, if the > CRC32 class (module) is part of a java.util.zip package (i.e. the output > of llvm-link), what happens? Is the entire package loaded providing more > than my program needs? Is the runtime linker smart enough to locate the > CRC32 module in the optimized package output? Is it smart enough to load > only the one module needed? Does the notion of module inside a module > even exist?There are multiple different ways to approach these questions depending on what we want to do and what the priorities are. There are several good solutions, but for now, everything needs to be statically linked. I expect this to change over the next month or so.> P.S. Bug 114 (-Wold-style-cast) has a patch ready.I realize that. I will get to it this afternoon, I'm still catching up from PLDI. -Chris -- http://llvm.cs.uiuc.edu/ http://www.nondot.org/~sabre/Projects/
On Sun, 2003-11-16 at 11:17, Chris Lattner wrote:> No, it's all or nothing. Once linked, they cannot be seperated (easily). > However, especially when using the JIT, there is little overhead for > running a gigantic program that only has 1% of the functions in it ever > executed...Perhaps in the general case, but what if its running on an embedded system and the "gigantic program" causes an out-of-memory condition?> There are multiple different ways to approach these questions depending on > what we want to do and what the priorities are. There are several good > solutions, but for now, everything needs to be statically linked. I > expect this to change over the next month or so.When you have time, I'd like to hear what you're planning in this area as it will directly effect how I build my compiler and VM. Thanks, Reid. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20031116/ab214fee/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20031116/ab214fee/attachment.sig>