Another usage case, slightly different than your #1:
6. Dynamic code generation (not interactive with respect to the source code)
- app generates code IR which is compiled as needed for execution
(important difference: no waiting for a typing human in the loop,
so very different expectations about responsiveness and bottlenecks)
- compilation speed IS critical (because many such compilations may be
necessary and the app itself may be interactive, even if the source code
entry is not)
- retained memory use is critical -- ok to use much mem while JITing, but
once
we've reduced it to callable machine code function, we want to reclaim
as much as possible of the memory needed by LLVM.
- app is multithreaded and does not want to lock on a single MCJIT --
ok to have multiple MCJIT's, one per app thread, needing to run
simultaneously.
I'm currently using the old JIT, but would happily switch to MCJIT if its
speed (especially ability to lazily compile functions as the old one can) were
to improve and memory was as good.
(Here's the GitHub page for the project I'm thinking of:
https://github.com/imageworks/OpenShadingLanguage/)
On Dec 9, 2013, at 11:35 AM, "Kaylor, Andrew" <andrew.kaylor at
intel.com> wrote:
> Below is an outline of various usage models for MCJIT that I put together
based on conversations at last month's LLVM Developer Meeting. If
you're using or thinking about using MCJIT and your use case doesn't
seem to fit in one of the categories below then either I didn't talk to you
or I didn't understand what you're doing.
>
> In any case, I'd like to see this get worked into a shape suitable for
inclusion in the LLVM documentation. I imagine it serving as a guide both to
those who are new to using MCJIT and to those who are developing and maintaining
MCJIT. If you're using MCJIT the latter (yes, the latter) case is
particularly important to you right now as having your use case properly
represented in this document is the best way to ensure that it is adequately
considered when changes are made to MCJIT and when the decision is made as to
when we are ready to deprecate the old JIT engine (probably in the 3.5 release,
BTW).
>
> So here's what I'm asking for: if you are currently using MCJIT or
considering using MCJIT, can you please find the use case that best fits your
program and comment on how well the outline describes it. If you understand
what I'm saying below but you see something that is missing, please let me
know. If you aren't sure what I'm saying or you don't know how
MCJIT might address your particular issues, please let me know that too. If you
think my outline is too sketchy and you need me to elaborate before you can
provide meaningful feedback, please let me know about that. If you think
it's the best piece of documentation you've read all year and you
can't wait to read it again, that's good information too.
>
> Thanks in advance for any and all feedback.
>
> -Andy
>
>
------------------------------------------------------------------------------------------
>
> Models for MCJIT use
>
> 1. Interactive dynamic code generation
> - user types code which is compiled as needed for execution
> - example: Kaleidoscope
> - compilation speed probably isn't critical
> - use one MCJIT instance with many modules
> - create new modules on compilation
> - MCJIT handles linking between modules
> - external references still need prototypes
> - we can at least provide a module pass to automate it
> - memory overhead may be an issue but MCJIT can fix that
> - see model 2 for pre-defined library
> - if processing a large script pre-optimize before passing modules to
MCJIT
>
> 2. Code generation for external target execution
> - client generates code to be injected into an external process
> - example: LLDB expression evaluation
> - target may be another local or remote
> - target architecture may not match host architecture
> - may use one or more instances of MCJIT (client preference)
> - MCJIT handles address remapping on request
> - custom memory manager handles code/data transfer
> - speed/memory requirements may vary
>
> 3. Large pre-defined module compilation and execution
> - code/IR is loaded from disk and prepared for execution
> - example: Intel(R) OpenCL SDK
> - compilation speed matters but isn't critical
> - initial startup time is somewhat important
> - execution speed is critical
> - memory consumption isn't an issue
> - tool integration may be important
> - use one MCJIT instance with multiple (but usually) few modules
> - use object caching for commonly used code
> - for very large, sparsely used libraries pre-link modules
> - object and archive support may be useful
>
> 4. Hot function replacement
> - client uses MCJIT to optimize frequently executed code
> - example: WebKit
> - compilation time is not critical
> - execution speed is critical
> - steady state memory consumption is very important
> - client handles pre-JIT interpretation/execution
> - MCJIT instances may be created as needed
> - custom memory manager transfers code memory ownership after
compilation
> - MCJIT instance is deleted when no longer needed
> - client handles function replacement and lifetime management
>
> 5. On demand "one-time" execution
> - client provides a library of code which is used by small, disposable
functions
> - example: database query?
> - initial load time isn't important
> - execution time is critical
> - if library code is fixed, load as shared library
> - if library code must be generated use a separate instance of MCJIT to
hold the library
> - this instance can support multiple modules
> - use a custom memory manager to link with functions in this module
> - object caching and archive support may be useful in this case
> - if inlining/lto is more important than compile time keep library in an
IR module and pre-link just before invoking MCJIT
> - create one instance of MCJIT as needed and destroy after execution
>
--
Larry Gritz
lg at larrygritz.com
Greetings, Larry Gritz wrote:> Another usage case, slightly different than your #1: > > 6. Dynamic code generation (not interactive with respect to the source code) > - app generates code IR which is compiled as needed for execution > (important difference: no waiting for a typing human in the loop, > so very different expectations about responsiveness and bottlenecks) > - compilation speed IS critical (because many such compilations may be > necessary and the app itself may be interactive, even if the source code > entry is not) > - retained memory use is critical -- ok to use much mem while JITing, but once > we've reduced it to callable machine code function, we want to reclaim > as much as possible of the memory needed by LLVM. > - app is multithreaded and does not want to lock on a single MCJIT -- > ok to have multiple MCJIT's, one per app thread, needing to run > simultaneously.For what it's worth, the above describes my usage case as well. (With the exception that at present the app uses only a single JIT thread.) Regards, Bill
Same here, in fact I'd imagine #6 is the case for many DSLs. v/r, Josh On Mon, Dec 9, 2013 at 4:24 PM, Bill Kelly <billk at cts.com> wrote:> Greetings, > > Larry Gritz wrote: > > Another usage case, slightly different than your #1: > > > > 6. Dynamic code generation (not interactive with respect to the source > code) > > - app generates code IR which is compiled as needed for execution > > (important difference: no waiting for a typing human in the loop, > > so very different expectations about responsiveness and > bottlenecks) > > - compilation speed IS critical (because many such compilations may be > > necessary and the app itself may be interactive, even if the source > code > > entry is not) > > - retained memory use is critical -- ok to use much mem while JITing, > but once > > we've reduced it to callable machine code function, we want to > reclaim > > as much as possible of the memory needed by LLVM. > > - app is multithreaded and does not want to lock on a single MCJIT -- > > ok to have multiple MCJIT's, one per app thread, needing to run > > simultaneously. > > For what it's worth, the above describes my usage case as well. (With the > exception that at present the app uses only a single JIT thread.) > > > Regards, > > Bill > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20131209/2033c5e1/attachment.html>
Hi Larry,
Thanks for the detailed description.
Have you done any speed and memory tests with MCJIT? I'm curious how far it
is from meeting your needs.
-Andy
-----Original Message-----
From: Larry Gritz [mailto:lg at larrygritz.com]
Sent: Monday, December 09, 2013 12:28 PM
To: LLVM Developers Mailing List
Cc: Kaylor, Andrew
Subject: Re: [RFC] MCJIT usage models
Another usage case, slightly different than your #1:
6. Dynamic code generation (not interactive with respect to the source code)
- app generates code IR which is compiled as needed for execution
(important difference: no waiting for a typing human in the loop,
so very different expectations about responsiveness and bottlenecks)
- compilation speed IS critical (because many such compilations may be
necessary and the app itself may be interactive, even if the source code
entry is not)
- retained memory use is critical -- ok to use much mem while JITing, but
once
we've reduced it to callable machine code function, we want to reclaim
as much as possible of the memory needed by LLVM.
- app is multithreaded and does not want to lock on a single MCJIT --
ok to have multiple MCJIT's, one per app thread, needing to run
simultaneously.
I'm currently using the old JIT, but would happily switch to MCJIT if its
speed (especially ability to lazily compile functions as the old one can) were
to improve and memory was as good.
(Here's the GitHub page for the project I'm thinking of:
https://github.com/imageworks/OpenShadingLanguage/)
On Dec 9, 2013, at 11:35 AM, "Kaylor, Andrew" <andrew.kaylor at
intel.com> wrote:
> Below is an outline of various usage models for MCJIT that I put together
based on conversations at last month's LLVM Developer Meeting. If
you're using or thinking about using MCJIT and your use case doesn't
seem to fit in one of the categories below then either I didn't talk to you
or I didn't understand what you're doing.
>
> In any case, I'd like to see this get worked into a shape suitable for
inclusion in the LLVM documentation. I imagine it serving as a guide both to
those who are new to using MCJIT and to those who are developing and maintaining
MCJIT. If you're using MCJIT the latter (yes, the latter) case is
particularly important to you right now as having your use case properly
represented in this document is the best way to ensure that it is adequately
considered when changes are made to MCJIT and when the decision is made as to
when we are ready to deprecate the old JIT engine (probably in the 3.5 release,
BTW).
>
> So here's what I'm asking for: if you are currently using MCJIT or
considering using MCJIT, can you please find the use case that best fits your
program and comment on how well the outline describes it. If you understand
what I'm saying below but you see something that is missing, please let me
know. If you aren't sure what I'm saying or you don't know how
MCJIT might address your particular issues, please let me know that too. If you
think my outline is too sketchy and you need me to elaborate before you can
provide meaningful feedback, please let me know about that. If you think
it's the best piece of documentation you've read all year and you
can't wait to read it again, that's good information too.
>
> Thanks in advance for any and all feedback.
>
> -Andy
>
>
------------------------------------------------------------------------------------------
>
> Models for MCJIT use
>
> 1. Interactive dynamic code generation
> - user types code which is compiled as needed for execution
> - example: Kaleidoscope
> - compilation speed probably isn't critical
> - use one MCJIT instance with many modules
> - create new modules on compilation
> - MCJIT handles linking between modules
> - external references still need prototypes
> - we can at least provide a module pass to automate it
> - memory overhead may be an issue but MCJIT can fix that
> - see model 2 for pre-defined library
> - if processing a large script pre-optimize before passing modules to
MCJIT
>
> 2. Code generation for external target execution
> - client generates code to be injected into an external process
> - example: LLDB expression evaluation
> - target may be another local or remote
> - target architecture may not match host architecture
> - may use one or more instances of MCJIT (client preference)
> - MCJIT handles address remapping on request
> - custom memory manager handles code/data transfer
> - speed/memory requirements may vary
>
> 3. Large pre-defined module compilation and execution
> - code/IR is loaded from disk and prepared for execution
> - example: Intel(R) OpenCL SDK
> - compilation speed matters but isn't critical
> - initial startup time is somewhat important
> - execution speed is critical
> - memory consumption isn't an issue
> - tool integration may be important
> - use one MCJIT instance with multiple (but usually) few modules
> - use object caching for commonly used code
> - for very large, sparsely used libraries pre-link modules
> - object and archive support may be useful
>
> 4. Hot function replacement
> - client uses MCJIT to optimize frequently executed code
> - example: WebKit
> - compilation time is not critical
> - execution speed is critical
> - steady state memory consumption is very important
> - client handles pre-JIT interpretation/execution
> - MCJIT instances may be created as needed
> - custom memory manager transfers code memory ownership after
compilation
> - MCJIT instance is deleted when no longer needed
> - client handles function replacement and lifetime management
>
> 5. On demand "one-time" execution
> - client provides a library of code which is used by small, disposable
functions
> - example: database query?
> - initial load time isn't important
> - execution time is critical
> - if library code is fixed, load as shared library
> - if library code must be generated use a separate instance of MCJIT to
hold the library
> - this instance can support multiple modules
> - use a custom memory manager to link with functions in this module
> - object caching and archive support may be useful in this case
> - if inlining/lto is more important than compile time keep library in an
IR module and pre-link just before invoking MCJIT
> - create one instance of MCJIT as needed and destroy after execution
>
--
Larry Gritz
lg at larrygritz.com