llvm dev - Nov 2017 - storing MBB MCSymbol in custom section

Dear llvm-dev-list,

I have created my own custom section to be added at the end into a
binary upon compilation which contains address of all basic blocks. As
the final address of the basic block is not known until link time, I
collect the MCSymbol* Symbol Values per BB in a temp array and at the
in the custom section and emit it (emitSymbolValue) into my section
within EmitEndOfAsmFile()

I have found people looking at slightly different uses of the basic
block address but running into the same problem as me when compiling.
$clang calc_pi.c -o calc_pi
fatal error: error in backend: Undefined temporary symbol

However when only compiling (incompletely via -S),
e.g. $clang calc_pi.c -o calc_pi -S ,
it does compile with the expected outputs and the basic block labels
properly generated in my section. I get a file with proper labels

e.g.
__custom_section:
    .long   19
    .quad   .LBB0_0
    .long   1
    .quad   .LBB0_1
    .long   28
    .quad   .LBB0_2
    .long   3
... etc.

TLDR: compiles correctly, will not link successfully to make a binary
because the symbol of the basic block in the .text section doesn't
exist. How do I ensure the Basic Block Symbol does not get destroyed
until my section is read in as well during linking?

Followup Question: How do I ensure that after my section has been
fleshed out with addresses, that I may then destroy those symbol
values as intended.

I feel it has something to do with the liveness of the basic blocks
symbols however I do not know where they are "destroyed" to prevent
this from happening to allow my custom section containing the basic
block labels to also be "converted to addresses" as in its original
function scope.

Sincerely,

Chris

Hi, 


I don't think file is actually "compiling fine", the error message
indicates that it is failing while trying to write the object file.

Looking at the assembly file I think you are emitting the section name but I
don't see any labels inside your section (unless you mean __custome_section
is a label). If your Basic block have no labels I think whatever object writer
you use will not create symbols for it. Perhaps you should look around that
area.


-----Original Message-----
From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of K
Jelesnianski via llvm-dev
Sent: Monday, November 27, 2017 5:28 PM
To: llvm-dev at lists.llvm.org
Subject: [llvm-dev] storing MBB MCSymbol in custom section

Dear llvm-dev-list,

I have created my own custom section to be added at the end into a binary upon
compilation which contains address of all basic blocks. As the final address of
the basic block is not known until link time, I collect the MCSymbol* Symbol
Values per BB in a temp array and at the in the custom section and emit it
(emitSymbolValue) into my section within EmitEndOfAsmFile()

I have found people looking at slightly different uses of the basic block
address but running into the same problem as me when compiling.
$clang calc_pi.c -o calc_pi
fatal error: error in backend: Undefined temporary symbol

However when only compiling (incompletely via -S), e.g. $clang calc_pi.c -o
calc_pi -S , it does compile with the expected outputs and the basic block
labels properly generated in my section. I get a file with proper labels

e.g.
__custom_section:
    .long   19
    .quad   .LBB0_0
    .long   1
    .quad   .LBB0_1
    .long   28
    .quad   .LBB0_2
    .long   3
... etc.

TLDR: compiles correctly, will not link successfully to make a binary because
the symbol of the basic block in the .text section doesn't exist. How do I
ensure the Basic Block Symbol does not get destroyed until my section is read in
as well during linking?

Followup Question: How do I ensure that after my section has been fleshed out
with addresses, that I may then destroy those symbol values as intended.

I feel it has something to do with the liveness of the basic blocks symbols
however I do not know where they are "destroyed" to prevent this from
happening to allow my custom section containing the basic block labels to also
be "converted to addresses" as in its original function scope.

Sincerely,

Chris
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Dear Sameer,

My assembly file shows that my new section (__custom_section) is being
created and populated with the basic block labels (.quad .LBB0_0,
.LBB0_1, ... etc) and other statistical data I am inputting (.long
19). If I compile without adding these basic block labels (.LBB0_0,
etc) my application compiles, links, and runs successfully as the
vanilla application would. Are you saying .LBB0_0, etc. is not an
actual basic block label used by LLVM? Inspecting assembly of other
parts of the file, I see branches reference these "things" (e.g. jge
.LBB0_2)

Thanks for the response.

Sincerely,

Chris

On Mon, Nov 27, 2017 at 9:57 PM, Sameer AbuAsal <sabuasal at
codeaurora.org> wrote:
> Hi,
>
>
> I don't think file is actually "compiling fine", the error
message indicates that it is failing while trying to write the object file.
>
> Looking at the assembly file I think you are emitting the section name but
I don't see any labels inside your section (unless you mean
__custome_section is a label). If your Basic block have no labels I think
whatever object writer you use will not create symbols for it. Perhaps you
should look around that area.
>
>
> -----Original Message-----
> From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of K
Jelesnianski via llvm-dev
> Sent: Monday, November 27, 2017 5:28 PM
> To: llvm-dev at lists.llvm.org
> Subject: [llvm-dev] storing MBB MCSymbol in custom section
>
> Dear llvm-dev-list,
>
> I have created my own custom section to be added at the end into a binary
upon compilation which contains address of all basic blocks. As the final
address of the basic block is not known until link time, I collect the MCSymbol*
Symbol Values per BB in a temp array and at the in the custom section and emit
it (emitSymbolValue) into my section within EmitEndOfAsmFile()
>
> I have found people looking at slightly different uses of the basic block
address but running into the same problem as me when compiling.
> $clang calc_pi.c -o calc_pi
> fatal error: error in backend: Undefined temporary symbol
>
> However when only compiling (incompletely via -S), e.g. $clang calc_pi.c -o
calc_pi -S , it does compile with the expected outputs and the basic block
labels properly generated in my section. I get a file with proper labels
>
> e.g.
> __custom_section:
>     .long   19
>     .quad   .LBB0_0
>     .long   1
>     .quad   .LBB0_1
>     .long   28
>     .quad   .LBB0_2
>     .long   3
> ... etc.
>
> TLDR: compiles correctly, will not link successfully to make a binary
because the symbol of the basic block in the .text section doesn't exist.
How do I ensure the Basic Block Symbol does not get destroyed until my section
is read in as well during linking?
>
> Followup Question: How do I ensure that after my section has been fleshed
out with addresses, that I may then destroy those symbol values as intended.
>
> I feel it has something to do with the liveness of the basic blocks symbols
however I do not know where they are "destroyed" to prevent this from
happening to allow my custom section containing the basic block labels to also
be "converted to addresses" as in its original function scope.
>
> Sincerely,
>
> Chris
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

Dear All,

Problem Solved! I'll reply in case some looks for something similar in
the future. Artem pointed out:

On Tue, Nov 28, 2017 at 10:52 AM, Tamazov, Artem <Artem.Tamazov at
amd.com> wrote:
> AFAIK the symbols starting with ".L" are not exported and thus
inaccessible during linking.
> There a special hack for that somewhere.
>
> BR artem//
>
Indeed this is true, after discussing with teammates, these Basic
block symbols are local labels (e.g. .LBB0_0, .LBB0_1 ...) are not
exported and not defined any more once computing the symbol table
giving error
"Undefined temporary symbol" in function
ELFObjectWriter::computeSymbolTable

The work around I came up with the help of my teammate was to create a
new symbol, borrowing the Basic Blocks name with a custom prefix and
emiting a label for that newly created symbol. This was implemented in
EmitFunctionBody(). Then, adding this label to my custom section.
Compilation and linking worked giving preserving program behavior and
my custom section contained the correct address of each functions
basic blocks when inspecting the disassembly.

Thank you to Artem and Sameer for your help super much!

Sincerely,

Chris

On Mon, Nov 27, 2017 at 9:57 PM, Sameer AbuAsal <sabuasal at
codeaurora.org> wrote:
> Hi,
>
>
> I don't think file is actually "compiling fine", the error
message indicates that it is failing while trying to write the object file.
>
> Looking at the assembly file I think you are emitting the section name but
I don't see any labels inside your section (unless you mean
__custome_section is a label). If your Basic block have no labels I think
whatever object writer you use will not create symbols for it. Perhaps you
should look around that area.
>
>
> -----Original Message-----
> From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of K
Jelesnianski via llvm-dev
> Sent: Monday, November 27, 2017 5:28 PM
> To: llvm-dev at lists.llvm.org
> Subject: [llvm-dev] storing MBB MCSymbol in custom section
>
> Dear llvm-dev-list,
>
> I have created my own custom section to be added at the end into a binary
upon compilation which contains address of all basic blocks. As the final
address of the basic block is not known until link time, I collect the MCSymbol*
Symbol Values per BB in a temp array and at the in the custom section and emit
it (emitSymbolValue) into my section within EmitEndOfAsmFile()
>
> I have found people looking at slightly different uses of the basic block
address but running into the same problem as me when compiling.
> $clang calc_pi.c -o calc_pi
> fatal error: error in backend: Undefined temporary symbol
>
> However when only compiling (incompletely via -S), e.g. $clang calc_pi.c -o
calc_pi -S , it does compile with the expected outputs and the basic block
labels properly generated in my section. I get a file with proper labels
>
> e.g.
> __custom_section:
>     .long   19
>     .quad   .LBB0_0
>     .long   1
>     .quad   .LBB0_1
>     .long   28
>     .quad   .LBB0_2
>     .long   3
> ... etc.
>
> TLDR: compiles correctly, will not link successfully to make a binary
because the symbol of the basic block in the .text section doesn't exist.
How do I ensure the Basic Block Symbol does not get destroyed until my section
is read in as well during linking?
>
> Followup Question: How do I ensure that after my section has been fleshed
out with addresses, that I may then destroy those symbol values as intended.
>
> I feel it has something to do with the liveness of the basic blocks symbols
however I do not know where they are "destroyed" to prevent this from
happening to allow my custom section containing the basic block labels to also
be "converted to addresses" as in its original function scope.
>
> Sincerely,
>
> Chris
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>