llvm dev - Sep 2019 - Questions after playing around with KaleidoscopeJIT (With source files)

Hello LLVM people,

after finishing Chapter 1 and 2 of the KaleidoscopeJIT tutorial, I started to
play around with the code and now, I have even more questions than before. I
hope that the people reading this could help me with it, to improve my
understanding about the LLVM and the JIT.
I have also the source code and binaries (Windows) available but because I
don't know how the mailing list treats attachments (like .zip) I decided to
only attach the source files - hoping it will help >o<

---

1.) Resolving undefined references
I wonder what the best practice is to resolve symbols that are not defined in my
llvm::Module.
For that purpose I created "CM_ExternalConstant", that is using
"extern const int planschiValue;" and compiled it with Clang. (See
CM_ExternalConstant.cpp)

"planschiValue" is not defined in the module, so it will be
unreferenced when I jit it. What is the best way to do this? I used 3 different
ways:

1.) I went over the llvm::Module before adding it to the layers. I searched for
the symbol "?planschiValue@@3HB" and called
"replaceAllUsesWith" to replace the symbol with the correct address.
(See PlanschbeckenJIT.cpp:31)
2.) I set my own GeneratorFunction and provided addresses for the symbols (See
PlanschbeckenJIT.cpp:95)
3.) I can use this->es.getMainJITDylib().define

Each way worked fine. But I felt that way 2.) was the most annoying way, because
I had to provide many other addresses. So what way is the best? Or can I
'freely' choose?
Is it possible to have multiple GeneratorFunctions? I would like to use the
'default' DynamicLibrarySearchGenerator plus my own lookup function.
Would it also be possible to replace the uses of "?planschiValue@@3HB"
directly with the actuall value of "plaschiValue"? Because it is a
constant?

2.) Loading multiple modules
I created "CM_Orc", which has a function "int helloOrc()" -
"CM_ExternalConstant" has the same function, but a different
implementation. (See CM_Orc.cpp and CM_ExternalConstant.cpp) Adding both modules
to the JIT will lead to an error message:
"Failure value returned from cantFail wrapped call
UNREACHABLE executed at c:\program
files\llvm\include\llvm\support\error.h:708!"
I always thought that in this situation, one of the functions would be
renamed...

3.) Loading multiple modules - controlled resolving
Finally I created "CM_PlanschiValue" this file defines a value for
"planschiValue" that was referenced in
"CM_ExternalConstant". (See CM_ PlanschiValue.cpp and
CM_ExternalConstant.cpp) Surprisingly, if I wrote:
"const int planschiValue = 543210;"
Clang would not generate code for it.
But when writing:
"extern const int planschiValue = 543210;" it worked. Why? Why can
Clang optimize away the code, when it could be referenced by a different module?
Like it was!

I loaded "CM_ExternalConstant" and "CM_PlanschiValue"
together, while using my own GeneratorFunction. But there was no need any more
to provide an address for "?planschiValue@@3HB". Obviously, the two
modules where linked - which is correct. But what, if I don't want that for
whatever reason? Would I have to rename the referenced value in the llvm::Module
before adding the second module? Or should I run instead two different JITs?
The idea behind this is... what should I do if I have multiple modules, that are
not meant to be used together but should run in the same process.

4.) Clang and JIT optimization
If I compile my modules with Clang and all optimizations turned on - do I still
need the TransformLayer in my JIT?

5.) Mangling names
Does Clang or the LLVM provide a function to mangle from the string "const
int planschiValue;" to "?planschiValue@@3HB"? Or vice versa?

---

I know that this are a lot questions and that I'm not good in explaining
stuff - but I really hope you guys can help me to understand everything better.

Kind greetings and many thanks
Björn
Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr.
DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus
Bode, Heiko Lampert, Takashi Nagano, Takeshi Fukushima. Junichi Tajika
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190925/9ddf8f88/attachment.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: CM_ExternalConstant.cpp
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190925/9ddf8f88/attachment.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: CM_Orc.cpp
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190925/9ddf8f88/attachment-0001.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: CM_PlanschiValue.cpp
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190925/9ddf8f88/attachment-0002.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: PlanschbeckenOrcJIT.cpp
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190925/9ddf8f88/attachment-0003.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: PlanschbeckenJIT.cpp
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190925/9ddf8f88/attachment-0004.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: PlanschbeckenJIT.h
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190925/9ddf8f88/attachment.h>

(+Lang, ORC leader)

I don't know much about most of the questions myself, though for (5) -
Clang does map between code and mangled names, but it doesn't provide a
simple entry-point to that. (mostly because it's not often useful in
isolation) - you can compile "const int foo;" to LLVM IR and
you'll see the
mangled name there. The narrowest piece of code that gives just the mangled
name would accept the Clang AST as input and operate on that, not on a
string of source code.

On Wed, Sep 25, 2019 at 2:37 AM Gaier, Bjoern via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hello LLVM people,
>
>
>
> after finishing Chapter 1 and 2 of the KaleidoscopeJIT tutorial, I started
> to play around with the code and now, I have even more questions than
> before. I hope that the people reading this could help me with it, to
> improve my understanding about the LLVM and the JIT.
>
> I have also the source code and binaries (Windows) available but because I
> don't know how the mailing list treats attachments (like .zip) I
decided to
> only attach the source files - hoping it will help >o<
>
>
>
> ---
>
>
>
> 1.) Resolving undefined references
>
> I wonder what the best practice is to resolve symbols that are not defined
> in my llvm::Module.
>
> For that purpose I created "CM_ExternalConstant", that is using
"extern
> const int planschiValue;" and compiled it with Clang. (See
> CM_ExternalConstant.cpp)
>
>
>
> "planschiValue" is not defined in the module, so it will be
unreferenced
> when I jit it. What is the best way to do this? I used 3 different ways:
>
>
>
> 1.) I went over the llvm::Module before adding it to the layers. I
> searched for the symbol "?planschiValue@@3HB" and called
> "replaceAllUsesWith" to replace the symbol with the correct
address. (See
> PlanschbeckenJIT.cpp:31)
>
> 2.) I set my own GeneratorFunction and provided addresses for the symbols
> (See PlanschbeckenJIT.cpp:95)
>
> 3.) I can use this->es.getMainJITDylib().define
>
>
>
> Each way worked fine. But I felt that way 2.) was the most annoying way,
> because I had to provide many other addresses. So what way is the best? Or
> can I 'freely' choose?
>
> Is it possible to have multiple GeneratorFunctions? I would like to use
> the 'default' DynamicLibrarySearchGenerator plus my own lookup
function.
>
> Would it also be possible to replace the uses of
"?planschiValue@@3HB"
> directly with the actuall value of "plaschiValue"? Because it is
a constant?
>
>
>
> 2.) Loading multiple modules
>
> I created "CM_Orc", which has a function "int
helloOrc()" -
> "CM_ExternalConstant" has the same function, but a different
> implementation. (See CM_Orc.cpp and CM_ExternalConstant.cpp) Adding both
> modules to the JIT will lead to an error message:
>
> "Failure value returned from cantFail wrapped call
>
> UNREACHABLE executed at c:\program
> files\llvm\include\llvm\support\error.h:708!"
>
> I always thought that in this situation, one of the functions would be
> renamed...
>
>
>
> 3.) Loading multiple modules - controlled resolving
>
> Finally I created "CM_PlanschiValue" this file defines a value
for
> "planschiValue" that was referenced in
"CM_ExternalConstant". (See CM_
> PlanschiValue.cpp and CM_ExternalConstant.cpp) Surprisingly, if I wrote:
>
> "const int planschiValue = 543210;"
>
> Clang would not generate code for it.
>
> But when writing:
>
> "extern const int planschiValue = 543210;" it worked. Why? Why
can Clang
> optimize away the code, when it could be referenced by a different module?
> Like it was!
>
>
>
> I loaded "CM_ExternalConstant" and "CM_PlanschiValue"
together, while
> using my own GeneratorFunction. But there was no need any more to provide
> an address for "?planschiValue@@3HB". Obviously, the two modules
where
> linked - which is correct. But what, if I don't want that for whatever
> reason? Would I have to rename the referenced value in the llvm::Module
> before adding the second module? Or should I run instead two different
JITs?
>
> The idea behind this is... what should I do if I have multiple modules,
> that are not meant to be used together but should run in the same process.
>
>
>
> 4.) Clang and JIT optimization
>
> If I compile my modules with Clang and all optimizations turned on - do I
> still need the TransformLayer in my JIT?
>
>
>
> 5.) Mangling names
>
> Does Clang or the LLVM provide a function to mangle from the string
"const
> int planschiValue;" to "?planschiValue@@3HB"? Or vice versa?
>
>
>
> ---
>
>
>
> I know that this are a lot questions and that I'm not good in
explaining
> stuff - but I really hope you guys can help me to understand everything
> better.
>
>
>
> Kind greetings and many thanks
>
> Björn
> Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816,
> USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert
> Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Takeshi Fukushima.
> Junichi Tajika
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190925/9002e567/attachment-0001.html>

Thank you David!

ORC leader is actually a pretty cool title. Anyway – what is the Clang AST?
Maybe I could try converting from a string to that AST myself? Currently I use
the way you described, by getting the name from the LLVM IR.

From: David Blaikie <dblaikie at gmail.com>
Sent: 25 September 2019 17:45
To: Gaier, Bjoern <Bjoern.Gaier at horiba.com>; Lang Hames <lhames at
gmail.com>
Cc: llvm-dev <llvm-dev at lists.llvm.org>
Subject: Re: [llvm-dev] Questions after playing around with KaleidoscopeJIT
(With source files)

(+Lang, ORC leader)

I don't know much about most of the questions myself, though for (5) - Clang
does map between code and mangled names, but it doesn't provide a simple
entry-point to that. (mostly because it's not often useful in isolation) -
you can compile "const int foo;" to LLVM IR and you'll see the
mangled name there. The narrowest piece of code that gives just the mangled name
would accept the Clang AST as input and operate on that, not on a string of
source code.

On Wed, Sep 25, 2019 at 2:37 AM Gaier, Bjoern via llvm-dev <llvm-dev at
lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
Hello LLVM people,

after finishing Chapter 1 and 2 of the KaleidoscopeJIT tutorial, I started to
play around with the code and now, I have even more questions than before. I
hope that the people reading this could help me with it, to improve my
understanding about the LLVM and the JIT.
I have also the source code and binaries (Windows) available but because I
don't know how the mailing list treats attachments (like .zip) I decided to
only attach the source files - hoping it will help >o<

---

1.) Resolving undefined references
I wonder what the best practice is to resolve symbols that are not defined in my
llvm::Module.
For that purpose I created "CM_ExternalConstant", that is using
"extern const int planschiValue;" and compiled it with Clang. (See
CM_ExternalConstant.cpp)

"planschiValue" is not defined in the module, so it will be
unreferenced when I jit it. What is the best way to do this? I used 3 different
ways:

1.) I went over the llvm::Module before adding it to the layers. I searched for
the symbol "?planschiValue@@3HB" and called
"replaceAllUsesWith" to replace the symbol with the correct address.
(See PlanschbeckenJIT.cpp:31)
2.) I set my own GeneratorFunction and provided addresses for the symbols (See
PlanschbeckenJIT.cpp:95)
3.) I can use this->es.getMainJITDylib().define

Each way worked fine. But I felt that way 2.) was the most annoying way, because
I had to provide many other addresses. So what way is the best? Or can I
'freely' choose?
Is it possible to have multiple GeneratorFunctions? I would like to use the
'default' DynamicLibrarySearchGenerator plus my own lookup function.
Would it also be possible to replace the uses of "?planschiValue@@3HB"
directly with the actuall value of "plaschiValue"? Because it is a
constant?

2.) Loading multiple modules
I created "CM_Orc", which has a function "int helloOrc()" -
"CM_ExternalConstant" has the same function, but a different
implementation. (See CM_Orc.cpp and CM_ExternalConstant.cpp) Adding both modules
to the JIT will lead to an error message:
"Failure value returned from cantFail wrapped call
UNREACHABLE executed at c:\program
files\llvm\include\llvm\support\error.h:708!"
I always thought that in this situation, one of the functions would be
renamed...

3.) Loading multiple modules - controlled resolving
Finally I created "CM_PlanschiValue" this file defines a value for
"planschiValue" that was referenced in
"CM_ExternalConstant". (See CM_ PlanschiValue.cpp and
CM_ExternalConstant.cpp) Surprisingly, if I wrote:
"const int planschiValue = 543210;"
Clang would not generate code for it.
But when writing:
"extern const int planschiValue = 543210;" it worked. Why? Why can
Clang optimize away the code, when it could be referenced by a different module?
Like it was!

I loaded "CM_ExternalConstant" and "CM_PlanschiValue"
together, while using my own GeneratorFunction. But there was no need any more
to provide an address for "?planschiValue@@3HB". Obviously, the two
modules where linked - which is correct. But what, if I don't want that for
whatever reason? Would I have to rename the referenced value in the llvm::Module
before adding the second module? Or should I run instead two different JITs?
The idea behind this is... what should I do if I have multiple modules, that are
not meant to be used together but should run in the same process.

4.) Clang and JIT optimization
If I compile my modules with Clang and all optimizations turned on - do I still
need the TransformLayer in my JIT?

5.) Mangling names
Does Clang or the LLVM provide a function to mangle from the string "const
int planschiValue;" to "?planschiValue@@3HB"? Or vice versa?

---

I know that this are a lot questions and that I'm not good in explaining
stuff - but I really hope you guys can help me to understand everything better.

Kind greetings and many thanks
Björn
Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr.
DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus
Bode, Heiko Lampert, Takashi Nagano, Takeshi Fukushima. Junichi Tajika
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr.
DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus
Bode, Heiko Lampert, Takashi Nagano, Takeshi Fukushima. Junichi Tajika
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190926/9b3f5b14/attachment.html>

Hi Gaier,

(2) - I'd imagine that this is more or less the expected behavior.
Function redefinition like that isn't legal in regular C++ land. If
you'd like to make the second definition shadow the old one though
that's a more interesting problem.

One idea that Patrick Nappa had is to name each function
"functionName_{count}" then keep a map between the function's
name,
and its count. When you add a function, increment the count and add it
to the JIT with that name. To look it up, check the map for its most
recent name and look that up.

(4) - AFIK Clang uses the same optimization passes as you might choose
to add to the JIT. There is a really great sackoverflow question that
covers the optimization passes that Clang uses for various
optimization flags here:

https://stackoverflow.com/questions/15548023/clang-optimization-levels

I'd imagine that you'd want to run those passes in the JIT because
that way you can control when things are optimized and likely speed up
compilation, but I'd be interested to see what you end up deciding on.

Also, as a side question, I'd be interested to know how you're using
clang to compile the C++ code into modules. I've been working on a
similar project and have found getting errors out of the parser
properly to be pretty difficult. If you've managed to solve that
problem, I'd love to hear about how you worked it out. Otherwise I
found Cling's incremental parsing code to be pretty helpful:

https://github.com/root-project/cling/blob/master/lib/Interpreter/IncrementalParser.cpp#L741

Good luck,
Zeke

On Wed, Sep 25, 2019 at 2:37 AM Gaier, Bjoern via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> Hello LLVM people,
>
>
>
> after finishing Chapter 1 and 2 of the KaleidoscopeJIT tutorial, I started
to play around with the code and now, I have even more questions than before. I
hope that the people reading this could help me with it, to improve my
understanding about the LLVM and the JIT.
>
> I have also the source code and binaries (Windows) available but because I
don't know how the mailing list treats attachments (like .zip) I decided to
only attach the source files - hoping it will help >o<
>
>
>
> ---
>
>
>
> 1.) Resolving undefined references
>
> I wonder what the best practice is to resolve symbols that are not defined
in my llvm::Module.
>
> For that purpose I created "CM_ExternalConstant", that is using
"extern const int planschiValue;" and compiled it with Clang. (See
CM_ExternalConstant.cpp)
>
>
>
> "planschiValue" is not defined in the module, so it will be
unreferenced when I jit it. What is the best way to do this? I used 3 different
ways:
>
>
>
> 1.) I went over the llvm::Module before adding it to the layers. I searched
for the symbol "?planschiValue@@3HB" and called
"replaceAllUsesWith" to replace the symbol with the correct address.
(See PlanschbeckenJIT.cpp:31)
>
> 2.) I set my own GeneratorFunction and provided addresses for the symbols
(See PlanschbeckenJIT.cpp:95)
>
> 3.) I can use this->es.getMainJITDylib().define
>
>
>
> Each way worked fine. But I felt that way 2.) was the most annoying way,
because I had to provide many other addresses. So what way is the best? Or can I
'freely' choose?
>
> Is it possible to have multiple GeneratorFunctions? I would like to use the
'default' DynamicLibrarySearchGenerator plus my own lookup function.
>
> Would it also be possible to replace the uses of
"?planschiValue@@3HB" directly with the actuall value of
"plaschiValue"? Because it is a constant?
>
>
>
> 2.) Loading multiple modules
>
> I created "CM_Orc", which has a function "int
helloOrc()" - "CM_ExternalConstant" has the same function, but a
different implementation. (See CM_Orc.cpp and CM_ExternalConstant.cpp) Adding
both modules to the JIT will lead to an error message:
>
> "Failure value returned from cantFail wrapped call
>
> UNREACHABLE executed at c:\program
files\llvm\include\llvm\support\error.h:708!"
>
> I always thought that in this situation, one of the functions would be
renamed...
>
>
>
> 3.) Loading multiple modules - controlled resolving
>
> Finally I created "CM_PlanschiValue" this file defines a value
for "planschiValue" that was referenced in
"CM_ExternalConstant". (See CM_ PlanschiValue.cpp and
CM_ExternalConstant.cpp) Surprisingly, if I wrote:
>
> "const int planschiValue = 543210;"
>
> Clang would not generate code for it.
>
> But when writing:
>
> "extern const int planschiValue = 543210;" it worked. Why? Why
can Clang optimize away the code, when it could be referenced by a different
module? Like it was!
>
>
>
> I loaded "CM_ExternalConstant" and "CM_PlanschiValue"
together, while using my own GeneratorFunction. But there was no need any more
to provide an address for "?planschiValue@@3HB". Obviously, the two
modules where linked - which is correct. But what, if I don't want that for
whatever reason? Would I have to rename the referenced value in the llvm::Module
before adding the second module? Or should I run instead two different JITs?
>
> The idea behind this is... what should I do if I have multiple modules,
that are not meant to be used together but should run in the same process.
>
>
>
> 4.) Clang and JIT optimization
>
> If I compile my modules with Clang and all optimizations turned on - do I
still need the TransformLayer in my JIT?
>
>
>
> 5.) Mangling names
>
> Does Clang or the LLVM provide a function to mangle from the string
"const int planschiValue;" to "?planschiValue@@3HB"? Or vice
versa?
>
>
>
> ---
>
>
>
> I know that this are a lot questions and that I'm not good in
explaining stuff - but I really hope you guys can help me to understand
everything better.
>
>
>
> Kind greetings and many thanks
>
> Björn
>
> Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816,
USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert
Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Takeshi Fukushima. Junichi
Tajika
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Hello Zeke Medley,

(2) - In the past I used MCJIT and when such a clash(?) happened, then the JIT
automatically renamed one of the functions by adding a number to it, like you
suggested. So I was surprised, that this didn't happened again.
I like the idea with the map a lot! Still waiting for an answer to (1) that
would help me imagine the implementation for that, because I would tend to do it
on the llvm::Module.

(4) - I use Clang-Cl (Windows user here) and simply add to my compiler flags
"-Xclang -emit-llvm-bc" or without the -bc. I then use a LLVM function
to load and parse the module. I hope that helps a bit? I'm a noob with the
LLVM so I might misunderstood the question.

Thank you a lot for your answer and the link! This helped me to go some steps
forward.

Kind greetings
Björn

-----Original Message-----
From: Zeke Medley <zekemedley at gmail.com>
Sent: 26 September 2019 18:36
To: Gaier, Bjoern <Bjoern.Gaier at horiba.com>
Cc: llvm-dev <llvm-dev at lists.llvm.org>
Subject: Re: [llvm-dev] Questions after playing around with KaleidoscopeJIT
(With source files)

Hi Gaier,

(2) - I'd imagine that this is more or less the expected behavior.
Function redefinition like that isn't legal in regular C++ land. If
you'd like to make the second definition shadow the old one though
that's a more interesting problem.

One idea that Patrick Nappa had is to name each function
"functionName_{count}" then keep a map between the function's
name, and its count. When you add a function, increment the count and add it to
the JIT with that name. To look it up, check the map for its most recent name
and look that up.

(4) - AFIK Clang uses the same optimization passes as you might choose to add to
the JIT. There is a really great sackoverflow question that covers the
optimization passes that Clang uses for various optimization flags here:

https://stackoverflow.com/questions/15548023/clang-optimization-levels

I'd imagine that you'd want to run those passes in the JIT because that
way you can control when things are optimized and likely speed up compilation,
but I'd be interested to see what you end up deciding on.

Also, as a side question, I'd be interested to know how you're using
clang to compile the C++ code into modules. I've been working on a similar
project and have found getting errors out of the parser properly to be pretty
difficult. If you've managed to solve that problem, I'd love to hear
about how you worked it out. Otherwise I found Cling's incremental parsing
code to be pretty helpful:

https://github.com/root-project/cling/blob/master/lib/Interpreter/IncrementalParser.cpp#L741

Good luck,
Zeke

On Wed, Sep 25, 2019 at 2:37 AM Gaier, Bjoern via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>
> Hello LLVM people,
>
>
>
> after finishing Chapter 1 and 2 of the KaleidoscopeJIT tutorial, I started
to play around with the code and now, I have even more questions than before. I
hope that the people reading this could help me with it, to improve my
understanding about the LLVM and the JIT.
>
> I have also the source code and binaries (Windows) available but
> because I don't know how the mailing list treats attachments (like
> .zip) I decided to only attach the source files - hoping it will help
> >o<
>
>
>
> ---
>
>
>
> 1.) Resolving undefined references
>
> I wonder what the best practice is to resolve symbols that are not defined
in my llvm::Module.
>
> For that purpose I created "CM_ExternalConstant", that is using
> "extern const int planschiValue;" and compiled it with Clang.
(See
> CM_ExternalConstant.cpp)
>
>
>
> "planschiValue" is not defined in the module, so it will be
unreferenced when I jit it. What is the best way to do this? I used 3 different
ways:
>
>
>
> 1.) I went over the llvm::Module before adding it to the layers. I
> searched for the symbol "?planschiValue@@3HB" and called
> "replaceAllUsesWith" to replace the symbol with the correct
address.
> (See PlanschbeckenJIT.cpp:31)
>
> 2.) I set my own GeneratorFunction and provided addresses for the
> symbols (See PlanschbeckenJIT.cpp:95)
>
> 3.) I can use this->es.getMainJITDylib().define
>
>
>
> Each way worked fine. But I felt that way 2.) was the most annoying way,
because I had to provide many other addresses. So what way is the best? Or can I
'freely' choose?
>
> Is it possible to have multiple GeneratorFunctions? I would like to use the
'default' DynamicLibrarySearchGenerator plus my own lookup function.
>
> Would it also be possible to replace the uses of
"?planschiValue@@3HB" directly with the actuall value of
"plaschiValue"? Because it is a constant?
>
>
>
> 2.) Loading multiple modules
>
> I created "CM_Orc", which has a function "int
helloOrc()" - "CM_ExternalConstant" has the same function, but a
different implementation. (See CM_Orc.cpp and CM_ExternalConstant.cpp) Adding
both modules to the JIT will lead to an error message:
>
> "Failure value returned from cantFail wrapped call
>
> UNREACHABLE executed at c:\program
files\llvm\include\llvm\support\error.h:708!"
>
> I always thought that in this situation, one of the functions would be
renamed...
>
>
>
> 3.) Loading multiple modules - controlled resolving
>
> Finally I created "CM_PlanschiValue" this file defines a value
for "planschiValue" that was referenced in
"CM_ExternalConstant". (See CM_ PlanschiValue.cpp and
CM_ExternalConstant.cpp) Surprisingly, if I wrote:
>
> "const int planschiValue = 543210;"
>
> Clang would not generate code for it.
>
> But when writing:
>
> "extern const int planschiValue = 543210;" it worked. Why? Why
can Clang optimize away the code, when it could be referenced by a different
module? Like it was!
>
>
>
> I loaded "CM_ExternalConstant" and "CM_PlanschiValue"
together, while using my own GeneratorFunction. But there was no need any more
to provide an address for "?planschiValue@@3HB". Obviously, the two
modules where linked - which is correct. But what, if I don't want that for
whatever reason? Would I have to rename the referenced value in the llvm::Module
before adding the second module? Or should I run instead two different JITs?
>
> The idea behind this is... what should I do if I have multiple modules,
that are not meant to be used together but should run in the same process.
>
>
>
> 4.) Clang and JIT optimization
>
> If I compile my modules with Clang and all optimizations turned on - do I
still need the TransformLayer in my JIT?
>
>
>
> 5.) Mangling names
>
> Does Clang or the LLVM provide a function to mangle from the string
"const int planschiValue;" to "?planschiValue@@3HB"? Or vice
versa?
>
>
>
> ---
>
>
>
> I know that this are a lot questions and that I'm not good in
explaining stuff - but I really hope you guys can help me to understand
everything better.
>
>
>
> Kind greetings and many thanks
>
> Björn
>
> Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816,
> USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr.
> Robert Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Takeshi
> Fukushima. Junichi Tajika
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr.
DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus
Bode, Heiko Lampert, Takashi Nagano, Takeshi Fukushima. Junichi Tajika