Hey Andy, Yep I've tested some non-trivial examples with loads of dependencies, both code and data, global, local and external symbol resolution etc.. Actually this was truly a piece of cake, nothing to do, the memory manager is working really nicely so far as I can tell. Relocations to sections are all working as expected (aside from previously mentioned ARM issue which is probably just something that I'm doing wrong) with all global symbol relocs managed persistently by the MM between object injections. All in all it just works ;) I had to make a few minor adjustments to things like the LLParser for forward dependencies but overall really simple stuff. There certainly are some section management issues that will need to be addressed, but I don't see any major hurdles there. I was going to take a look into this next week? The biggest issue for multi-module is probably going to be client side not LLVM side, although this has not been a huge problem for me as most of this bookkeeping is already managed "client side" in extempore. I'm happy to send you code although it might be more useful for me to write a followup email outlining exactly what changes were made and then let the experts decide how best to proceed ;) Tomorrows a little hectic but I'll try to send a note through on Monday. Cheers, Andrew. On Sat, Feb 16, 2013 at 8:13 AM, Kaylor, Andrew <andrew.kaylor at intel.com>wrote:> This is great news.**** > > ** ** > > Do you have any dependencies between your modules? For instance, one > calling a function in another? If so, how did you handle that?**** > > ** ** > > Any chance you could share some code snippets or the relevant portions?*** > * > > ** ** > > -Andy**** > > ** ** > > *From:* Andrew Sorensen [mailto:digegoo at gmail.com] > *Sent:* Thursday, February 14, 2013 11:48 PM > > *To:* Kaylor, Andrew > *Cc:* llvmdev at cs.uiuc.edu > *Subject:* Re: [LLVMdev] MCJIT and Lazy Compilation**** > > ** ** > > OK, so I have some *preliminary* results, which are on the whole quite > encouraging!**** > > ** ** > > I haven't had a great deal of time, but I have managed to get Extempore up > and **** > > running with function (actually lexical closures so composed of quite a > bit of additional**** > > guff) level compilation using Andy's multi module suggestion. I also have > on-the-fly **** > > recompilation of existing closures working (caveats below) so from an > end-user **** > > perspective this means that Extempore appears functionally equivalent with > MCJIT **** > > and the old legacy JIT - hot-swapping audio signal processing code > on-the-fly using **** > > MCJIT for example.**** > > ** ** > > Firstly multi-module definitely proved to be considerably easier than > attempting to hack**** > > solutions for incremental *monolithic* module builds - which I also > investigated.**** > > ** ** > > So the only major obstacle that I have run into so far are page > permissions in relation**** > > to code relocations. I have a *safe* hack which is to toggle section > permissions between**** > > rw and exec/ro in-between new object injections - however this is > obviously problematic **** > > for code that is executing concurrently (i.e. secondary threads). I also > have an *unsafe***** > > hack, (purely for experimentation :-) whereby exec sections are left rw, > and although **** > > very evil it works for test purposes (i.e. the audio example mentioned > above). These **** > > solutions are obviously both inappropriate and I will investigate a *real* > solution when **** > > I find some time.**** > > ** ** > > Also I didn't bother to implement section erasure, at the moment I'm just > allocating **** > > new sections for each compile regardless of whether the new code replaces > existing**** > > functionality. Having said that I don't see this as much of an issue, I > was just to**** > > lazy to bother implementing it. I'll check this when I have some further > free time.**** > > ** ** > > FYI this is all under x86. I did try to run under ARM but bombed out on > an assertion error **** > > in the ARM ELF relocation code - specifically assert((*TargetPtr & > 0x000F0FFF) == 0);**** > > I assume this is a result of something evil that I have done but I haven't > yet had time to **** > > investigate any further. Again I'll let you know when I have some more > time.**** > > ** ** > > Just a quick heads up but In general my initial thoughts are that MCJIT is > really not **** > > that far off.**** > > ** ** > > Cheers,**** > > Andrew.**** > > ** ** > > ** ** > > ** ** > > **** > > ** ** > > **** > > ** ** > > ** ** > > ** ** > > ** ** > > On Fri, Feb 8, 2013 at 7:36 AM, Kaylor, Andrew <andrew.kaylor at intel.com> > wrote:**** > > That’s awesome!**** > > **** > > I think at this point having people try out various approaches and seeing > what works and what doesn’t is our biggest need in this area. Please do > keep me informed about what you find out.**** > > **** > > -Andy**** > > **** > > *From:* Andrew Sorensen [mailto:digegoo at gmail.com] > *Sent:* Wednesday, February 06, 2013 4:33 PM > *To:* Kaylor, Andrew > *Cc:* llvmdev at cs.uiuc.edu > *Subject:* Re: [LLVMdev] MCJIT and Lazy Compilation**** > > **** > > Thanks for the update Andy.**** > > **** > > I'm very happy to be involved in anyway that is helpful. If you would > like me to test ideas, or contribute to further discussions, then please > let me know.**** > > **** > > I currently have extempore running nicely with MCJIT for the "monolithic" > case and am working on various LLVM hacks to better understand the issues > involved with non-monolithic approaches - in particular I'm starting with > your multi-module approach. I will report back when (and if) I have > something useful to contribute.**** > > **** > > Cheers,**** > > Andrew.**** > > **** > > On Wed, Feb 6, 2013 at 4:08 AM, Kaylor, Andrew <andrew.kaylor at intel.com> > wrote:**** > > Hi Andrew,**** > > **** > > I was about to write a belated reply to this message (sorry for the > delay), but then I realized that pretty much everything useful that I have > to say on the subject is contained in this message (which is in a thread > Albert Graef already linked to):**** > > **** > > https://groups.google.com/d/msg/llvm-dev/Rk9cWdRX0Wg/Fa1Mn6cyS9UJ**** > > **** > > Generally, I do hope that MCJIT will be capable of replacing the old JIT > someday soon, though obviously it cannot do so until it provides equivalent > functionality. I doubt it will ever be a “drop-in” replacement, but I hope > that minimal rework will be needed. Most significantly, as can be seen in > earlier discussions, things will need to be made Module-centric rather than > Function-centric. It ought to be possible to write a utility class that > takes a monolithic Module and breaks it up into sub-Modules for individual > functions, but I think that would need to happen outside of the MCJIT > engine because not all clients would want that kind of granularity.**** > > **** > > There’s definitely a lot of work to be done here to get this right, and > hopefully we’ll get active participation in any design discussions to make > sure the solution meets everyone’s needs. I don’t have a time table for > this right now. I will file a Bugzilla report as soon as the LLVM server > is ready.**** > > **** > > -Andy**** > > **** > > *From:* llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] *On > Behalf Of *Andrew Sorensen > *Sent:* Thursday, January 31, 2013 7:56 PM > *To:* llvmdev at cs.uiuc.edu > *Subject:* [LLVMdev] MCJIT and Lazy Compilation**** > > **** > > Does anyone have a roadmap for MCJIT with what I think people are **** > > calling lazy compilation.**** > > **** > > Is this even on the cards?**** > > **** > > I spent the last few hours moving my project (extempore.moso.com.au) **** > > over to MCJIT (particularly for ARM), and am a little horrified to > discover **** > > no ability to compile, and just as importantly to recompile, at a function > level. **** > > This is absolutely mandatory for my project. **** > > **** > > I have been looking enviously at MCJIT's ARM+DWARF support for a **** > > couple of years and was under the misapprehension that MCJIT was **** > > attempting to be a *drop-in* replacement for JIT. So I wasn't overly**** > > concerned about the primary JIT being largely neglected. This is obviously > **** > > my fault, I wasn't paying close enough attention.**** > > **** > > I am now wondering what the LLVM project, in the large, plans regarding ** > ** > > just-in-time compilation moving forward. Is MCJIT the future, and**** > > if so what kind of roadmap is there to replicate current JIT > functionality. **** > > In my case in relation to function level (re)compilation.**** > > **** > > I appreciate everyones efforts, and that we all have our own agendas.**** > > I'm just trying to put my own roadmap in place.**** > > **** > > Cheers,**** > > Andrew.**** > > **** > > ** ** >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130216/789e2b63/attachment.html>
Hi Andy/Albert,
Sorry for the slow reply, my day job caught up with me.
Two bits of progress. (a) MCJIT is working nicely for non-trivial
examples in Extempore on x86 and ARM, and (b) the page
permissions are now RO again. For your amusement a *very*
cheesy video of Extempore running on-the-fly with MCJIT on an
ARM Pandaboard. Viewer discretion is advised!
https://vimeo.com/60407237
Here is the overview of changes I promised a couple of weeks back.
These comments are based on the 3.3 trunk of about 3 weeks ago.
-- RuntimeDyld --
Relocations needs to be cleared between
each emitObject.
New method required: clearRelocations
void RuntimeDyldImpl::clearRelocations() {
ExternalSymbolRelocations.clear();
Relocations.clear();
}
void RuntimeDyld::clearRelocations() {
Dyld->clearRelocations();
}
-- RuntimeDyldImpl --
void clearRelocations();
-- SectionMemoryManager --
Removed option in allocateSection to use existing 'free' memory
regions - i.e. now always allocate new memory. This means that once a
module is emitted and its memory permissions are applied we don't have
to touch it again. This means we don't have to set exec sections
writable before a new emit. I'm sure there is a nicer way to achieve
this.
-- MCJIT --
MCJIT currently takes a single Module and only supports a single call
to emitObject. A number of assertion and conditional checks enforce
this. These checks can all be removed (assertions and conditionals),
making M obsolete.
At the moment there are a number of call sites for emitObject. I
removed all of them and replaced them with a single emitObject call
site accessible through the extant, but unused,
recompileAndRelinkFunction method. This then becomes the clients
access point to compile each individual module. This is obviously a
hack to maintain the integrity of the existing API (i.e. bad name,
evil type munging F->M etc..).
void *MCJIT::recompileAndRelinkFunction(Function *F) {
emitObject((Module*) F);
finalizeObject();
return NULL;
}
FinalizeObject should not call resolveRelocations. Instead it should
call the *new* clearRelocations.
void MCJIT::finalizeObject() {
// New Dyld call clearRelocations
Dyld.clearRelocations();
// Set page permissions.
MemMgr->applyPermissions();
}
At the moment I am just leaking allocated sections in SMM. i.e.
void MCJIT::freeMachineCodeForFunction(Function *F) {
dbgs() << "free machine code not yet supported in MCJIT\n";
return;
}
But it should be relatively straight forward to maintain some kind of
Module -> SMM Section map. Hotswapping currently works fine because
the relocations all update as expected. So this is *just* a leakage
problem.
-- AsmParser/LLParser --
Some forward refs in the LLParser need fixing. I'm working directly
from IR though, so C++ API people will need a different fix.
-------------------------
I might have missed a few things as it has been a couple of weeks
since I looked at the code but that is the general drift. As
previously mentioned there really is not much required to make all of
this function well for the individual module case.
Of course this does not address the issues that clients may face
moving from JIT to MCJIT as a result of these changes. However, I
expect that many have similar solutions to Extempore, and should not
have any major problems moving to a multi-module approach. Albert do
you see this being much of a problem for Pure?
There are obviously some things that will need fixing.
1) The current single Module case may need to be retained??
2) We need a more appropriate call than recompileAndRelinkFunction
3) We need proper section reclamation/erasure
4) Better solution for permissions re. freemem in section allocation.
Andy, aside from these 4 points does everything here sound OK to you,
and if so how would you like to proceed. I can put *some* time in but
you'll need to give me some direction about what you would like done.
Cheers,
Andrew.
On Sat, Feb 16, 2013 at 5:26 PM, Andrew Sorensen <digegoo at gmail.com>
wrote:
> Hey Andy,
>
> Yep I've tested some non-trivial examples with loads of dependencies,
> both code and data, global, local and external symbol resolution etc..
>
> Actually this was truly a piece of cake, nothing to do, the memory manager
> is working really nicely so far as I can tell. Relocations to sections
> are all working
> as expected (aside from previously mentioned ARM issue which is probably
> just
> something that I'm doing wrong) with all global symbol relocs managed
> persistently
> by the MM between object injections. All in all it just works ;) I had
> to make a few
> minor adjustments to things like the LLParser for forward dependencies but
> overall
> really simple stuff.
>
> There certainly are some section management issues that will need to be
> addressed,
> but I don't see any major hurdles there. I was going to take a look
into
> this next week?
>
> The biggest issue for multi-module is probably going to be client side not
> LLVM side,
> although this has not been a huge problem for me as most of this
> bookkeeping is
> already managed "client side" in extempore.
>
> I'm happy to send you code although it might be more useful for me to
> write a
> followup email outlining exactly what changes were made and then let the
> experts
> decide how best to proceed ;) Tomorrows a little hectic but I'll try
to
> send a note
> through on Monday.
>
>
> Cheers,
> Andrew.
>
>
>
>
>
>
> On Sat, Feb 16, 2013 at 8:13 AM, Kaylor, Andrew <andrew.kaylor at
intel.com>wrote:
>
>> This is great news.****
>>
>> ** **
>>
>> Do you have any dependencies between your modules? For instance, one
>> calling a function in another? If so, how did you handle that?****
>>
>> ** **
>>
>> Any chance you could share some code snippets or the relevant
portions?**
>> **
>>
>> ** **
>>
>> -Andy****
>>
>> ** **
>>
>> *From:* Andrew Sorensen [mailto:digegoo at gmail.com]
>> *Sent:* Thursday, February 14, 2013 11:48 PM
>>
>> *To:* Kaylor, Andrew
>> *Cc:* llvmdev at cs.uiuc.edu
>> *Subject:* Re: [LLVMdev] MCJIT and Lazy Compilation****
>>
>> ** **
>>
>> OK, so I have some *preliminary* results, which are on the whole quite
>> encouraging!****
>>
>> ** **
>>
>> I haven't had a great deal of time, but I have managed to get
Extempore
>> up and ****
>>
>> running with function (actually lexical closures so composed of quite a
>> bit of additional****
>>
>> guff) level compilation using Andy's multi module suggestion. I
also have
>> on-the-fly ****
>>
>> recompilation of existing closures working (caveats below) so from an
>> end-user ****
>>
>> perspective this means that Extempore appears functionally equivalent
>> with MCJIT ****
>>
>> and the old legacy JIT - hot-swapping audio signal processing code
>> on-the-fly using ****
>>
>> MCJIT for example.****
>>
>> ** **
>>
>> Firstly multi-module definitely proved to be considerably easier than
>> attempting to hack****
>>
>> solutions for incremental *monolithic* module builds - which I also
>> investigated.****
>>
>> ** **
>>
>> So the only major obstacle that I have run into so far are page
>> permissions in relation****
>>
>> to code relocations. I have a *safe* hack which is to toggle section
>> permissions between****
>>
>> rw and exec/ro in-between new object injections - however this is
>> obviously problematic ****
>>
>> for code that is executing concurrently (i.e. secondary threads). I
also
>> have an *unsafe*****
>>
>> hack, (purely for experimentation :-) whereby exec sections are left
rw,
>> and although ****
>>
>> very evil it works for test purposes (i.e. the audio example mentioned
>> above). These ****
>>
>> solutions are obviously both inappropriate and I will investigate a
>> *real* solution when ****
>>
>> I find some time.****
>>
>> ** **
>>
>> Also I didn't bother to implement section erasure, at the moment
I'm just
>> allocating ****
>>
>> new sections for each compile regardless of whether the new code
replaces
>> existing****
>>
>> functionality. Having said that I don't see this as much of an
issue, I
>> was just to****
>>
>> lazy to bother implementing it. I'll check this when I have some
further
>> free time.****
>>
>> ** **
>>
>> FYI this is all under x86. I did try to run under ARM but bombed out
on
>> an assertion error ****
>>
>> in the ARM ELF relocation code - specifically assert((*TargetPtr
&
>> 0x000F0FFF) == 0);****
>>
>> I assume this is a result of something evil that I have done but I
>> haven't yet had time to ****
>>
>> investigate any further. Again I'll let you know when I have some
more
>> time.****
>>
>> ** **
>>
>> Just a quick heads up but In general my initial thoughts are that MCJIT
>> is really not ****
>>
>> that far off.****
>>
>> ** **
>>
>> Cheers,****
>>
>> Andrew.****
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ****
>>
>> ** **
>>
>> ****
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> On Fri, Feb 8, 2013 at 7:36 AM, Kaylor, Andrew <andrew.kaylor at
intel.com>
>> wrote:****
>>
>> That’s awesome!****
>>
>> ****
>>
>> I think at this point having people try out various approaches and
seeing
>> what works and what doesn’t is our biggest need in this area. Please
do
>> keep me informed about what you find out.****
>>
>> ****
>>
>> -Andy****
>>
>> ****
>>
>> *From:* Andrew Sorensen [mailto:digegoo at gmail.com]
>> *Sent:* Wednesday, February 06, 2013 4:33 PM
>> *To:* Kaylor, Andrew
>> *Cc:* llvmdev at cs.uiuc.edu
>> *Subject:* Re: [LLVMdev] MCJIT and Lazy Compilation****
>>
>> ****
>>
>> Thanks for the update Andy.****
>>
>> ****
>>
>> I'm very happy to be involved in anyway that is helpful. If you
would
>> like me to test ideas, or contribute to further discussions, then
please
>> let me know.****
>>
>> ****
>>
>> I currently have extempore running nicely with MCJIT for the
"monolithic"
>> case and am working on various LLVM hacks to better understand the
issues
>> involved with non-monolithic approaches - in particular I'm
starting with
>> your multi-module approach. I will report back when (and if) I have
>> something useful to contribute.****
>>
>> ****
>>
>> Cheers,****
>>
>> Andrew.****
>>
>> ****
>>
>> On Wed, Feb 6, 2013 at 4:08 AM, Kaylor, Andrew <andrew.kaylor at
intel.com>
>> wrote:****
>>
>> Hi Andrew,****
>>
>> ****
>>
>> I was about to write a belated reply to this message (sorry for the
>> delay), but then I realized that pretty much everything useful that I
have
>> to say on the subject is contained in this message (which is in a
thread
>> Albert Graef already linked to):****
>>
>> ****
>>
>> https://groups.google.com/d/msg/llvm-dev/Rk9cWdRX0Wg/Fa1Mn6cyS9UJ****
>>
>> ****
>>
>> Generally, I do hope that MCJIT will be capable of replacing the old
JIT
>> someday soon, though obviously it cannot do so until it provides
equivalent
>> functionality. I doubt it will ever be a “drop-in” replacement, but I
hope
>> that minimal rework will be needed. Most significantly, as can be seen
in
>> earlier discussions, things will need to be made Module-centric rather
than
>> Function-centric. It ought to be possible to write a utility class
that
>> takes a monolithic Module and breaks it up into sub-Modules for
individual
>> functions, but I think that would need to happen outside of the MCJIT
>> engine because not all clients would want that kind of granularity.****
>>
>> ****
>>
>> There’s definitely a lot of work to be done here to get this right, and
>> hopefully we’ll get active participation in any design discussions to
make
>> sure the solution meets everyone’s needs. I don’t have a time table
for
>> this right now. I will file a Bugzilla report as soon as the LLVM
server
>> is ready.****
>>
>> ****
>>
>> -Andy****
>>
>> ****
>>
>> *From:* llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at
cs.uiuc.edu]
>> *On Behalf Of *Andrew Sorensen
>> *Sent:* Thursday, January 31, 2013 7:56 PM
>> *To:* llvmdev at cs.uiuc.edu
>> *Subject:* [LLVMdev] MCJIT and Lazy Compilation****
>>
>> ****
>>
>> Does anyone have a roadmap for MCJIT with what I think people are ****
>>
>> calling lazy compilation.****
>>
>> ****
>>
>> Is this even on the cards?****
>>
>> ****
>>
>> I spent the last few hours moving my project (extempore.moso.com.au)
****
>>
>> over to MCJIT (particularly for ARM), and am a little horrified to
>> discover ****
>>
>> no ability to compile, and just as importantly to recompile, at a
>> function level. ****
>>
>> This is absolutely mandatory for my project. ****
>>
>> ****
>>
>> I have been looking enviously at MCJIT's ARM+DWARF support for a
****
>>
>> couple of years and was under the misapprehension that MCJIT was ****
>>
>> attempting to be a *drop-in* replacement for JIT. So I wasn't
overly****
>>
>> concerned about the primary JIT being largely neglected. This is
obviously
>> ****
>>
>> my fault, I wasn't paying close enough attention.****
>>
>> ****
>>
>> I am now wondering what the LLVM project, in the large, plans regarding
*
>> ***
>>
>> just-in-time compilation moving forward. Is MCJIT the future, and****
>>
>> if so what kind of roadmap is there to replicate current JIT
>> functionality. ****
>>
>> In my case in relation to function level (re)compilation.****
>>
>> ****
>>
>> I appreciate everyones efforts, and that we all have our own
agendas.****
>>
>> I'm just trying to put my own roadmap in place.****
>>
>> ****
>>
>> Cheers,****
>>
>> Andrew.****
>>
>> ****
>>
>> ** **
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130309/1ab13d28/attachment.html>
Hi Andrew, the prototype looks great! Andy Kaylor is on vacation, so I'm not
sure if he'll see this thread until next week.
You definitely have MCJIT moving in the right direction, but the API you're
using (ExecutionEngine) has to be stretched a little bit to accomplish your
goals. In your implementation, do clients call recompileAndRelinkFunction() for
every module that is needed to satisfy inter-module dependencies (if so, can the
same thing be accomplished with the existing ExecutionEngine::addModule()?) or
do clients call it only to update ("hotswap") a module that has been
previously compiled but has since changed?
Off hand, it sounds like ExecutionEngine might need another function like:
updateModule(Module* M);
which does what your modified recompileAndRelinkFunction() does. It should be
easy to add any new functions you need to the API in ExecutionEngine.h as
virtuals (with default, empty, implementations in order to keep the old JIT
happy) and overrides in MCJIT that do the right thing.
Regarding tests, I'm thinking that the existing lli-based MCJIT integration
tests (tests/ExecutionEngine/MCJIT) are going to be harder to extend to support
multiple modules (with function dependencies, globals, etc) than the unit tests
(unittests/ExecutionEngine/MCJIT). Anyways, it sounds like you have already been
testing lots of cases, but I'm not sure if that's been done manually or
in some automatic fashion. There is a (commented-out) test in MCJITTests.cpp
that intends to test the case of dependencies between modules, but there's
probably lots of other cases that can be added too. In any case, I find that
writing unit tests is a big help to figure out what an API should look like.
Also, I believe there's an assumption (in at least one place) that
ExecutionEngines take ownership of Modules that are passed in. It sounds like
you removed it (from MCJIT at least) by getting rid of the field M and the
corresponding assertions, but I'm wondering if there's any other common
code in ExecutionEngine that makes the same assumption about a single module
whose ownership is transferred.
Can you elaborate on what you mean by "1) The current single Module case
may need to be retained??". I don't see why the single-module case
cannot be handled in exactly the same way as the multiple module cases.
Ashok (cc'd) has in the past looked at MCJIT memory managers, and
specifically at applying permissions, so he might have some more thoughts, and
I'll leave it to Andy K to review at the relocation stuff as that's his
domain of expertise!
At this point though, I think lots people are excited about MCJIT (specifically,
multiple modules with dependencies and hot-swapping) and would love to see a
patch when you have a moment to rebase it on top of current trunk. Don't
worry if the patch is incomplete; just seeing its design might help elicit some
more targeted comments!
Thanks,
Dan
From: Andrew Sorensen <digegoo at gmail.com<mailto:digegoo at
gmail.com>>
Date: Saturday, 9 March, 2013 4:26 AM
To: Andrew Kaylor <andrew.kaylor at intel.com<mailto:andrew.kaylor at
intel.com>>
Cc: LLVM List <llvmdev at cs.uiuc.edu<mailto:llvmdev at
cs.uiuc.edu>>
Subject: Re: [LLVMdev] MCJIT and Lazy Compilation
Hi Andy/Albert,
Sorry for the slow reply, my day job caught up with me.
Two bits of progress. (a) MCJIT is working nicely for non-trivial
examples in Extempore on x86 and ARM, and (b) the page
permissions are now RO again. For your amusement a *very*
cheesy video of Extempore running on-the-fly with MCJIT on an
ARM Pandaboard. Viewer discretion is advised!
https://vimeo.com/60407237
Here is the overview of changes I promised a couple of weeks back.
These comments are based on the 3.3 trunk of about 3 weeks ago.
-- RuntimeDyld --
Relocations needs to be cleared between
each emitObject.
New method required: clearRelocations
void RuntimeDyldImpl::clearRelocations() {
ExternalSymbolRelocations.clear();
Relocations.clear();
}
void RuntimeDyld::clearRelocations() {
Dyld->clearRelocations();
}
-- RuntimeDyldImpl --
void clearRelocations();
-- SectionMemoryManager --
Removed option in allocateSection to use existing 'free' memory
regions - i.e. now always allocate new memory. This means that once a
module is emitted and its memory permissions are applied we don't have
to touch it again. This means we don't have to set exec sections
writable before a new emit. I'm sure there is a nicer way to achieve
this.
-- MCJIT --
MCJIT currently takes a single Module and only supports a single call
to emitObject. A number of assertion and conditional checks enforce
this. These checks can all be removed (assertions and conditionals),
making M obsolete.
At the moment there are a number of call sites for emitObject. I
removed all of them and replaced them with a single emitObject call
site accessible through the extant, but unused,
recompileAndRelinkFunction method. This then becomes the clients
access point to compile each individual module. This is obviously a
hack to maintain the integrity of the existing API (i.e. bad name,
evil type munging F->M etc..).
void *MCJIT::recompileAndRelinkFunction(Function *F) {
emitObject((Module*) F);
finalizeObject();
return NULL;
}
FinalizeObject should not call resolveRelocations. Instead it should
call the *new* clearRelocations.
void MCJIT::finalizeObject() {
// New Dyld call clearRelocations
Dyld.clearRelocations();
// Set page permissions.
MemMgr->applyPermissions();
}
At the moment I am just leaking allocated sections in SMM. i.e.
void MCJIT::freeMachineCodeForFunction(Function *F) {
dbgs() << "free machine code not yet supported in MCJIT\n";
return;
}
But it should be relatively straight forward to maintain some kind of
Module -> SMM Section map. Hotswapping currently works fine because
the relocations all update as expected. So this is *just* a leakage
problem.
-- AsmParser/LLParser --
Some forward refs in the LLParser need fixing. I'm working directly
from IR though, so C++ API people will need a different fix.
-------------------------
I might have missed a few things as it has been a couple of weeks
since I looked at the code but that is the general drift. As
previously mentioned there really is not much required to make all of
this function well for the individual module case.
Of course this does not address the issues that clients may face
moving from JIT to MCJIT as a result of these changes. However, I
expect that many have similar solutions to Extempore, and should not
have any major problems moving to a multi-module approach. Albert do
you see this being much of a problem for Pure?
There are obviously some things that will need fixing.
1) The current single Module case may need to be retained??
2) We need a more appropriate call than recompileAndRelinkFunction
3) We need proper section reclamation/erasure
4) Better solution for permissions re. freemem in section allocation.
Andy, aside from these 4 points does everything here sound OK to you,
and if so how would you like to proceed. I can put *some* time in but
you'll need to give me some direction about what you would like done.
Cheers,
Andrew.
On Sat, Feb 16, 2013 at 5:26 PM, Andrew Sorensen <digegoo at
gmail.com<mailto:digegoo at gmail.com>> wrote:
Hey Andy,
Yep I've tested some non-trivial examples with loads of dependencies,
both code and data, global, local and external symbol resolution etc..
Actually this was truly a piece of cake, nothing to do, the memory manager
is working really nicely so far as I can tell. Relocations to sections are all
working
as expected (aside from previously mentioned ARM issue which is probably just
something that I'm doing wrong) with all global symbol relocs managed
persistently
by the MM between object injections. All in all it just works ;) I had to make
a few
minor adjustments to things like the LLParser for forward dependencies but
overall
really simple stuff.
There certainly are some section management issues that will need to be
addressed,
but I don't see any major hurdles there. I was going to take a look into
this next week?
The biggest issue for multi-module is probably going to be client side not LLVM
side,
although this has not been a huge problem for me as most of this bookkeeping is
already managed "client side" in extempore.
I'm happy to send you code although it might be more useful for me to write
a
followup email outlining exactly what changes were made and then let the experts
decide how best to proceed ;) Tomorrows a little hectic but I'll try to
send a note
through on Monday.
Cheers,
Andrew.
On Sat, Feb 16, 2013 at 8:13 AM, Kaylor, Andrew <andrew.kaylor at
intel.com<mailto:andrew.kaylor at intel.com>> wrote:
This is great news.
Do you have any dependencies between your modules? For instance, one calling a
function in another? If so, how did you handle that?
Any chance you could share some code snippets or the relevant portions?
-Andy
From: Andrew Sorensen [mailto:digegoo at gmail.com<mailto:digegoo at
gmail.com>]
Sent: Thursday, February 14, 2013 11:48 PM
To: Kaylor, Andrew
Cc: llvmdev at cs.uiuc.edu<mailto:llvmdev at cs.uiuc.edu>
Subject: Re: [LLVMdev] MCJIT and Lazy Compilation
OK, so I have some *preliminary* results, which are on the whole quite
encouraging!
I haven't had a great deal of time, but I have managed to get Extempore up
and
running with function (actually lexical closures so composed of quite a bit of
additional
guff) level compilation using Andy's multi module suggestion. I also have
on-the-fly
recompilation of existing closures working (caveats below) so from an end-user
perspective this means that Extempore appears functionally equivalent with MCJIT
and the old legacy JIT - hot-swapping audio signal processing code on-the-fly
using
MCJIT for example.
Firstly multi-module definitely proved to be considerably easier than attempting
to hack
solutions for incremental *monolithic* module builds - which I also
investigated.
So the only major obstacle that I have run into so far are page permissions in
relation
to code relocations. I have a *safe* hack which is to toggle section
permissions between
rw and exec/ro in-between new object injections - however this is obviously
problematic
for code that is executing concurrently (i.e. secondary threads). I also have
an *unsafe*
hack, (purely for experimentation :-) whereby exec sections are left rw, and
although
very evil it works for test purposes (i.e. the audio example mentioned above).
These
solutions are obviously both inappropriate and I will investigate a *real*
solution when
I find some time.
Also I didn't bother to implement section erasure, at the moment I'm
just allocating
new sections for each compile regardless of whether the new code replaces
existing
functionality. Having said that I don't see this as much of an issue, I was
just to
lazy to bother implementing it. I'll check this when I have some further
free time.
FYI this is all under x86. I did try to run under ARM but bombed out on an
assertion error
in the ARM ELF relocation code - specifically assert((*TargetPtr &
0x000F0FFF) == 0);
I assume this is a result of something evil that I have done but I haven't
yet had time to
investigate any further. Again I'll let you know when I have some more
time.
Just a quick heads up but In general my initial thoughts are that MCJIT is
really not
that far off.
Cheers,
Andrew.
On Fri, Feb 8, 2013 at 7:36 AM, Kaylor, Andrew <andrew.kaylor at
intel.com<mailto:andrew.kaylor at intel.com>> wrote:
That’s awesome!
I think at this point having people try out various approaches and seeing what
works and what doesn’t is our biggest need in this area. Please do keep me
informed about what you find out.
-Andy
From: Andrew Sorensen [mailto:digegoo at gmail.com<mailto:digegoo at
gmail.com>]
Sent: Wednesday, February 06, 2013 4:33 PM
To: Kaylor, Andrew
Cc: llvmdev at cs.uiuc.edu<mailto:llvmdev at cs.uiuc.edu>
Subject: Re: [LLVMdev] MCJIT and Lazy Compilation
Thanks for the update Andy.
I'm very happy to be involved in anyway that is helpful. If you would like
me to test ideas, or contribute to further discussions, then please let me know.
I currently have extempore running nicely with MCJIT for the
"monolithic" case and am working on various LLVM hacks to better
understand the issues involved with non-monolithic approaches - in particular
I'm starting with your multi-module approach. I will report back when (and
if) I have something useful to contribute.
Cheers,
Andrew.
On Wed, Feb 6, 2013 at 4:08 AM, Kaylor, Andrew <andrew.kaylor at
intel.com<mailto:andrew.kaylor at intel.com>> wrote:
Hi Andrew,
I was about to write a belated reply to this message (sorry for the delay), but
then I realized that pretty much everything useful that I have to say on the
subject is contained in this message (which is in a thread Albert Graef already
linked to):
https://groups.google.com/d/msg/llvm-dev/Rk9cWdRX0Wg/Fa1Mn6cyS9UJ
Generally, I do hope that MCJIT will be capable of replacing the old JIT someday
soon, though obviously it cannot do so until it provides equivalent
functionality. I doubt it will ever be a “drop-in” replacement, but I hope that
minimal rework will be needed. Most significantly, as can be seen in earlier
discussions, things will need to be made Module-centric rather than
Function-centric. It ought to be possible to write a utility class that takes a
monolithic Module and breaks it up into sub-Modules for individual functions,
but I think that would need to happen outside of the MCJIT engine because not
all clients would want that kind of granularity.
There’s definitely a lot of work to be done here to get this right, and
hopefully we’ll get active participation in any design discussions to make sure
the solution meets everyone’s needs. I don’t have a time table for this right
now. I will file a Bugzilla report as soon as the LLVM server is ready.
-Andy
From:llvmdev-bounces at cs.uiuc.edu<mailto:llvmdev-bounces at cs.uiuc.edu>
[mailto:llvmdev-bounces at cs.uiuc.edu<mailto:llvmdev-bounces at
cs.uiuc.edu>] On Behalf Of Andrew Sorensen
Sent: Thursday, January 31, 2013 7:56 PM
To: llvmdev at cs.uiuc.edu<mailto:llvmdev at cs.uiuc.edu>
Subject: [LLVMdev] MCJIT and Lazy Compilation
Does anyone have a roadmap for MCJIT with what I think people are
calling lazy compilation.
Is this even on the cards?
I spent the last few hours moving my project
(extempore.moso.com.au<http://extempore.moso.com.au>)
over to MCJIT (particularly for ARM), and am a little horrified to discover
no ability to compile, and just as importantly to recompile, at a function
level.
This is absolutely mandatory for my project.
I have been looking enviously at MCJIT's ARM+DWARF support for a
couple of years and was under the misapprehension that MCJIT was
attempting to be a *drop-in* replacement for JIT. So I wasn't overly
concerned about the primary JIT being largely neglected. This is obviously
my fault, I wasn't paying close enough attention.
I am now wondering what the LLVM project, in the large, plans regarding
just-in-time compilation moving forward. Is MCJIT the future, and
if so what kind of roadmap is there to replicate current JIT functionality.
In my case in relation to function level (re)compilation.
I appreciate everyones efforts, and that we all have our own agendas.
I'm just trying to put my own roadmap in place.
Cheers,
Andrew.