llvm dev - Aug 2020 - End-to-end -fembed-bitcode .llvmbc and .llvmcmd

Hello,

I'm trying to understand how .llvmbc and .llvmcmd fit into an end-to-end
story. From the RFC
<http://lists.llvm.org/pipermail/llvm-dev/2016-February/094851.html>, and
reading through the implementation, I'm piecing together that the goal was
to enable capturing IR right after clang and before passing it to
LLVM's optimization passes, as well as the command line options needed for
later compiling that IR to the same native object it was compiled to
originally (with the same compiler).

Here's what I don't understand: say you have a.o and b.o compiled with
-fembed-bitcode=all. They are linked into a binary called my_binary. How do
you re-create the corresponding IR for modules a and b (let's call them
a.bc and b.bc), and their corresponding command lines? From what I can
tell, the linker just concatenates the IR for a and b in my_binary's
.llvmbc, and the same for the command line in .llvmcmd. Is there a
separator maybe I missed? For .llvmcmd, I could see how *maybe* -cc1 could
be that separator, what about the .llvmbc part? The magic number?

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200824/ddd41ba3/attachment.html>

Hi Mircea

From the RFC you mentioned, that is a Darwin specific implementation, which
later got extended to support other targets. The main use case for the embed
bitcode option is to allow compiler passing intermediate IR and command flags in
the object file it produced for later use. For Darwin, it is used for bitcode
recompilation, and some might use it to achieve other goals.

In order to use this information properly, you needs to have tools that
understand the layout and sections for embedded bitcode. You can't just use
an ordinary linker, because like you said, an ELF linker will just append the
bitcode. Depending on what you are trying to achieve, you need to implement the
downstream tools, like linker, binary analysis tools, etc. to understand this
concept.

Steven
> On Aug 24, 2020, at 7:10 PM, Mircea Trofin via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Hello,
> 
> I'm trying to understand how .llvmbc and .llvmcmd fit into an
end-to-end story. From the RFC
<http://lists.llvm.org/pipermail/llvm-dev/2016-February/094851.html>, and
reading through the implementation, I'm piecing together that the goal was
to enable capturing IR right after clang and before passing it to LLVM's
optimization passes, as well as the command line options needed for later
compiling that IR to the same native object it was compiled to originally (with
the same compiler).
> 
> Here's what I don't understand: say you have a.o and b.o compiled
with -fembed-bitcode=all. They are linked into a binary called my_binary. How do
you re-create the corresponding IR for modules a and b (let's call them a.bc
and b.bc), and their corresponding command lines? From what I can tell, the
linker just concatenates the IR for a and b in my_binary's .llvmbc, and the
same for the command line in .llvmcmd. Is there a separator maybe I missed? For
.llvmcmd, I could see how *maybe* -cc1 could be that separator, what about the
.llvmbc part? The magic number?
> 
> Thanks!
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200827/1be735bc/attachment.html>