llvm dev - Dec 2013 - [LLVMdev] Making LLVM safer in out-of-memory situations

Hi Philip,

Thanks for the positive response from all of you!
 
> One question: How are you handling EOM?  Error return?  Custom region
allocator?
When running into an Out-of-memory situation we're currently only doing an
error return, i.e. the compilation fails, but does so without crashing the
process in which the compilation/jitting occurs. It is ok for us if llvm returns
with a catchable exception and unwinds all allocated memory correctly.

To increase stability for us we have already moved the main part of the
compilation to a separate process that may crash in case of an error without
doing much harm, i.e. does not crash the database. Therefore, we've
currently concentrating on specific components that still remain in the database
process, such as CodeLoader and VMCore, which is used for emitting IR code.
Although, of course, we're also interested in increasing the general
stability of the whole llvm w.r.t. error situations.

Best regards,
Philipp

-----Original Message-----
From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] On
Behalf Of Philip Reames
Sent: Freitag, 13. Dezember 2013 02:04
To: Gasiunas, Vaidas; LLVM Dev
Subject: Re: [LLVMdev] Making LLVM safer in out-of-memory situations

On 12/12/13 4:25 AM, Gasiunas, Vaidas wrote:
> Hello,
>
> Philipp Becker and me, Vaidas Gasiunas, are developers at SAP and part of a
team developing a C-like domain-specific language for the SAP HANA in-memory
database. We use LLVM as a backend to translate our language to native code,
primarily on x86-64 platforms. Our programs are created dynamically, compiled
and optimized in a running database. As a result of that we have special
requirements with respect to response time and safety. In particular, we have to
avoid long compile times and must deal with error situations like out-of-memory
without crashing or producing memory leaks in the compiler. The compiler
performance is especially important since that we must compile generated
functions which tend to be rather long - in the range of thousands of LOC per
function.
>
> To address these requirements we have developed a set of patches improving
performance and malfunction safety of certain compiler passes and would be
interested in contributing them at some point. Before proposing concrete
changes, we would like to know what the general interest is with respect to
making LLVM safer in out-of-memory situations.
I'm hugely in favor of the general direction.  Happy to help by 
reviewing changes and the like.

This type of work was on my long term todo list; I'm thrilled to see 
someone else doing it now.  :)

One question: How are you handling EOM?  Error return?  Custom region 
allocator?

Philip
_______________________________________________
LLVM Developers mailing list
LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

----- Original Message -----
> From: "Philipp Becker" <philipp.becker at sap.com>
> To: "Philip Reames" <listmail at philipreames.com>,
"LLVM Dev" <llvmdev at cs.uiuc.edu>
> Sent: Friday, December 13, 2013 6:55:59 AM
> Subject: Re: [LLVMdev] Making LLVM safer in out-of-memory situations
> 
> Hi Philip,
> 
> Thanks for the positive response from all of you!
>  
> > One question: How are you handling EOM?  Error return?  Custom
> > region allocator?
> 
> When running into an Out-of-memory situation we're currently only
> doing an error return, i.e. the compilation fails, but does so
> without crashing the process in which the compilation/jitting
> occurs. It is ok for us if llvm returns with a catchable exception
> and unwinds all allocated memory correctly.
Does this mean that you're using C++ exception handling to manage the
cleanup?

 -Hal
> 
> To increase stability for us we have already moved the main part of
> the compilation to a separate process that may crash in case of an
> error without doing much harm, i.e. does not crash the database.
> Therefore, we've currently concentrating on specific components that
> still remain in the database process, such as CodeLoader and VMCore,
> which is used for emitting IR code. Although, of course, we're also
> interested in increasing the general stability of the whole llvm
> w.r.t. error situations.
> 
> Best regards,
> Philipp
> 
> -----Original Message-----
> From: llvmdev-bounces at cs.uiuc.edu
> [mailto:llvmdev-bounces at cs.uiuc.edu] On Behalf Of Philip Reames
> Sent: Freitag, 13. Dezember 2013 02:04
> To: Gasiunas, Vaidas; LLVM Dev
> Subject: Re: [LLVMdev] Making LLVM safer in out-of-memory situations
> 
> On 12/12/13 4:25 AM, Gasiunas, Vaidas wrote:
> > Hello,
> >
> > Philipp Becker and me, Vaidas Gasiunas, are developers at SAP and
> > part of a team developing a C-like domain-specific language for
> > the SAP HANA in-memory database. We use LLVM as a backend to
> > translate our language to native code, primarily on x86-64
> > platforms. Our programs are created dynamically, compiled and
> > optimized in a running database. As a result of that we have
> > special requirements with respect to response time and safety. In
> > particular, we have to avoid long compile times and must deal with
> > error situations like out-of-memory without crashing or producing
> > memory leaks in the compiler. The compiler performance is
> > especially important since that we must compile generated
> > functions which tend to be rather long - in the range of thousands
> > of LOC per function.
> >
> > To address these requirements we have developed a set of patches
> > improving performance and malfunction safety of certain compiler
> > passes and would be interested in contributing them at some point.
> > Before proposing concrete changes, we would like to know what the
> > general interest is with respect to making LLVM safer in
> > out-of-memory situations.
> I'm hugely in favor of the general direction.  Happy to help by
> reviewing changes and the like.
> 
> This type of work was on my long term todo list; I'm thrilled to see
> someone else doing it now.  :)
> 
> One question: How are you handling EOM?  Error return?  Custom region
> allocator?
> 
> Philip
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 
-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory

Hi Hal,
> Does this mean that you're using C++ exception handling to manage the
cleanup?
No, not really. From the place where we're calling into llvm we are catching
all exceptions that may occur during compilation, but normally we do not add any
additional catch clauses into the llvm source itself. We mainly rely on correct
stack unwinding by destructors in llvm when an exception is thrown. In the cases
when it is not sufficient, we had to add some additional autopointers, an in
some cases implement additional unwind logic. In some cases we indeed had to add
some exception handling in destructors in cases where they allocate memory, but
such fixes are rather workarounds, because a correct solution would be to avoid
memory allocation in destructors in the first place.

Best regards,
Philipp

-----Original Message-----
From: Hal Finkel [mailto:hfinkel at anl.gov] 
Sent: Freitag, 13. Dezember 2013 14:32
To: Becker, Philipp
Cc: Philip Reames; LLVM Dev
Subject: Re: [LLVMdev] Making LLVM safer in out-of-memory situations

----- Original Message -----
> From: "Philipp Becker" <philipp.becker at sap.com>
> To: "Philip Reames" <listmail at philipreames.com>,
"LLVM Dev" <llvmdev at cs.uiuc.edu>
> Sent: Friday, December 13, 2013 6:55:59 AM
> Subject: Re: [LLVMdev] Making LLVM safer in out-of-memory situations
> 
> Hi Philip,
> 
> Thanks for the positive response from all of you!
>  
> > One question: How are you handling EOM?  Error return?  Custom
> > region allocator?
> 
> When running into an Out-of-memory situation we're currently only
> doing an error return, i.e. the compilation fails, but does so
> without crashing the process in which the compilation/jitting
> occurs. It is ok for us if llvm returns with a catchable exception
> and unwinds all allocated memory correctly.
Does this mean that you're using C++ exception handling to manage the
cleanup?

 -Hal
> 
> To increase stability for us we have already moved the main part of
> the compilation to a separate process that may crash in case of an
> error without doing much harm, i.e. does not crash the database.
> Therefore, we've currently concentrating on specific components that
> still remain in the database process, such as CodeLoader and VMCore,
> which is used for emitting IR code. Although, of course, we're also
> interested in increasing the general stability of the whole llvm
> w.r.t. error situations.
> 
> Best regards,
> Philipp
> 
> -----Original Message-----
> From: llvmdev-bounces at cs.uiuc.edu
> [mailto:llvmdev-bounces at cs.uiuc.edu] On Behalf Of Philip Reames
> Sent: Freitag, 13. Dezember 2013 02:04
> To: Gasiunas, Vaidas; LLVM Dev
> Subject: Re: [LLVMdev] Making LLVM safer in out-of-memory situations
> 
> On 12/12/13 4:25 AM, Gasiunas, Vaidas wrote:
> > Hello,
> >
> > Philipp Becker and me, Vaidas Gasiunas, are developers at SAP and
> > part of a team developing a C-like domain-specific language for
> > the SAP HANA in-memory database. We use LLVM as a backend to
> > translate our language to native code, primarily on x86-64
> > platforms. Our programs are created dynamically, compiled and
> > optimized in a running database. As a result of that we have
> > special requirements with respect to response time and safety. In
> > particular, we have to avoid long compile times and must deal with
> > error situations like out-of-memory without crashing or producing
> > memory leaks in the compiler. The compiler performance is
> > especially important since that we must compile generated
> > functions which tend to be rather long - in the range of thousands
> > of LOC per function.
> >
> > To address these requirements we have developed a set of patches
> > improving performance and malfunction safety of certain compiler
> > passes and would be interested in contributing them at some point.
> > Before proposing concrete changes, we would like to know what the
> > general interest is with respect to making LLVM safer in
> > out-of-memory situations.
> I'm hugely in favor of the general direction.  Happy to help by
> reviewing changes and the like.
> 
> This type of work was on my long term todo list; I'm thrilled to see
> someone else doing it now.  :)
> 
> One question: How are you handling EOM?  Error return?  Custom region
> allocator?
> 
> Philip
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 
-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory

On 12/13/13 4:55 AM, Becker, Philipp wrote:
> To increase stability for us we have already moved the main part of the
compilation to a separate process that may crash in case of an error without
doing much harm, i.e. does not crash the database.
Were there any interesting challenges that arose during this process?  
This seems to be an approach many folks are either taking or 
considering.  If there are things we could do to make this easier, it 
might be worth considering.
> Therefore, we've currently concentrating on specific components that
still remain in the database process, such as CodeLoader and VMCore, which is
used for emitting IR code. Although, of course, we're also interested in
increasing the general stability of the whole llvm w.r.t. error situations.
>
Understood.  If you have patches that need reviewed, I'd be happy to help.

Philip

>> To increase stability for us we have already moved the main part of the
compilation to a separate process that may crash in case of an error without
doing much harm, i.e. does not crash the database.
> Were there any interesting challenges that arose during this process?  
> This seems to be an approach many folks are either taking or 
> considering.  If there are things we could do to make this easier, it 
> might be worth considering.
After porting our project to LLVM3.1, we realized that we can use 
the MCJIT architecture to move compilation into a separate process, 
because it enables loading ELF objects  generated in another process. 
In fact, it worked as expected. It was really an important improvement 
for our use scenario. 

Maybe what was not so nice is that we had to patch RuntimeDyld to 
adapt it to our requirements. We had to extend it with a method for 
computing the total size of memory required for loading all sections 
required for execution, so that we can allocate one memory block for 
all sections, and another method to retrieve the address ranges of all 
loaded functions so that we have a mapping from addresses to function 
names.

Regards,
Vaidas

-----Original Message-----
From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] On
Behalf Of Philip Reames
Sent: Donnerstag, 19. Dezember 2013 18:32
To: Becker, Philipp; LLVM Dev
Subject: Re: [LLVMdev] Making LLVM safer in out-of-memory situations

On 12/13/13 4:55 AM, Becker, Philipp wrote:
> To increase stability for us we have already moved the main part of the
compilation to a separate process that may crash in case of an error without
doing much harm, i.e. does not crash the database.
Were there any interesting challenges that arose during this process?  
This seems to be an approach many folks are either taking or 
considering.  If there are things we could do to make this easier, it 
might be worth considering.
> Therefore, we've currently concentrating on specific components that
still remain in the database process, such as CodeLoader and VMCore, which is
used for emitting IR code. Although, of course, we're also interested in
increasing the general stability of the whole llvm w.r.t. error situations.
>
Understood.  If you have patches that need reviewed, I'd be happy to help.

Philip
_______________________________________________
LLVM Developers mailing list
LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev