llvm dev - Dec 2013 - [LLVMdev] Making LLVM safer in out-of-memory situations

Hi Hal,
> Does this mean that you're using C++ exception handling to manage the
cleanup?
No, not really. From the place where we're calling into llvm we are catching
all exceptions that may occur during compilation, but normally we do not add any
additional catch clauses into the llvm source itself. We mainly rely on correct
stack unwinding by destructors in llvm when an exception is thrown. In the cases
when it is not sufficient, we had to add some additional autopointers, an in
some cases implement additional unwind logic. In some cases we indeed had to add
some exception handling in destructors in cases where they allocate memory, but
such fixes are rather workarounds, because a correct solution would be to avoid
memory allocation in destructors in the first place.

Best regards,
Philipp

-----Original Message-----
From: Hal Finkel [mailto:hfinkel at anl.gov] 
Sent: Freitag, 13. Dezember 2013 14:32
To: Becker, Philipp
Cc: Philip Reames; LLVM Dev
Subject: Re: [LLVMdev] Making LLVM safer in out-of-memory situations

----- Original Message -----
> From: "Philipp Becker" <philipp.becker at sap.com>
> To: "Philip Reames" <listmail at philipreames.com>,
"LLVM Dev" <llvmdev at cs.uiuc.edu>
> Sent: Friday, December 13, 2013 6:55:59 AM
> Subject: Re: [LLVMdev] Making LLVM safer in out-of-memory situations
> 
> Hi Philip,
> 
> Thanks for the positive response from all of you!
>  
> > One question: How are you handling EOM?  Error return?  Custom
> > region allocator?
> 
> When running into an Out-of-memory situation we're currently only
> doing an error return, i.e. the compilation fails, but does so
> without crashing the process in which the compilation/jitting
> occurs. It is ok for us if llvm returns with a catchable exception
> and unwinds all allocated memory correctly.
Does this mean that you're using C++ exception handling to manage the
cleanup?

 -Hal
> 
> To increase stability for us we have already moved the main part of
> the compilation to a separate process that may crash in case of an
> error without doing much harm, i.e. does not crash the database.
> Therefore, we've currently concentrating on specific components that
> still remain in the database process, such as CodeLoader and VMCore,
> which is used for emitting IR code. Although, of course, we're also
> interested in increasing the general stability of the whole llvm
> w.r.t. error situations.
> 
> Best regards,
> Philipp
> 
> -----Original Message-----
> From: llvmdev-bounces at cs.uiuc.edu
> [mailto:llvmdev-bounces at cs.uiuc.edu] On Behalf Of Philip Reames
> Sent: Freitag, 13. Dezember 2013 02:04
> To: Gasiunas, Vaidas; LLVM Dev
> Subject: Re: [LLVMdev] Making LLVM safer in out-of-memory situations
> 
> On 12/12/13 4:25 AM, Gasiunas, Vaidas wrote:
> > Hello,
> >
> > Philipp Becker and me, Vaidas Gasiunas, are developers at SAP and
> > part of a team developing a C-like domain-specific language for
> > the SAP HANA in-memory database. We use LLVM as a backend to
> > translate our language to native code, primarily on x86-64
> > platforms. Our programs are created dynamically, compiled and
> > optimized in a running database. As a result of that we have
> > special requirements with respect to response time and safety. In
> > particular, we have to avoid long compile times and must deal with
> > error situations like out-of-memory without crashing or producing
> > memory leaks in the compiler. The compiler performance is
> > especially important since that we must compile generated
> > functions which tend to be rather long - in the range of thousands
> > of LOC per function.
> >
> > To address these requirements we have developed a set of patches
> > improving performance and malfunction safety of certain compiler
> > passes and would be interested in contributing them at some point.
> > Before proposing concrete changes, we would like to know what the
> > general interest is with respect to making LLVM safer in
> > out-of-memory situations.
> I'm hugely in favor of the general direction.  Happy to help by
> reviewing changes and the like.
> 
> This type of work was on my long term todo list; I'm thrilled to see
> someone else doing it now.  :)
> 
> One question: How are you handling EOM?  Error return?  Custom region
> allocator?
> 
> Philip
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 
-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory

On 12/13/13 6:47 AM, Becker, Philipp wrote:
> Hi Hal,
>
>> Does this mean that you're using C++ exception handling to manage
the cleanup?
> No, not really. From the place where we're calling into llvm we are
catching all exceptions that may occur during compilation, but normally we do
not add any additional catch clauses into the llvm source itself. We mainly rely
on correct stack unwinding by destructors in llvm when an exception is thrown.
In the cases when it is not sufficient, we had to add some additional
autopointers, an in some cases implement additional unwind logic. In some cases
we indeed had to add some exception handling in destructors in cases where they
allocate memory, but such fixes are rather workarounds, because a correct
solution would be to avoid memory allocation in destructors in the first place.
If I'm reading you correctly, you are relying on exception propagation 
and handler (destructors for local objects) execution. You have chosen 
not to add extra exception logic to LLVM itself, but are relying on the 
correctness of exception propagation within the code.  (The last two 
sentances are intended to be a restatement of what your message said.  
If I misunderstood, please correct me.)

Does this mean that you're compiling your build of LLVM with exceptions 
enabled?  By default, I believe LLVM is built without RTTI or exception 
support.

For the particular cases you mentioned with auto pointers and allocation 
in destructors, are these issues also present along existing error 
paths?  Or for that matter simply examples of bad coding practice?  If 
so, pushing back selected changes would be welcomed.  I'd be happy to 
help review.

Philip

Hi Philip,
> If I'm reading you correctly, you are relying on exception propagation 
> and handler (destructors for local objects) execution. You have chosen 
> not to add extra exception logic to LLVM itself, but are relying on the 
> correctness of exception propagation within the code.  (The last two 
> sentances are intended to be a restatement of what your message said.  
> If I misunderstood, please correct me.)
It was probably not completely correct to say that we did not extend 
the exception propagation in LLVM. In most of cases where malloc or 
other C allocation functions are called, we had to add a check for NULL
and throw std::bad_alloc. But these are a kind of straightforward fixes, 
that do not require much effort. 
> Does this mean that you're compiling your build of LLVM with exceptions
> enabled?  By default, I believe LLVM is built without RTTI or exception 
> support.
OK, I see. This explains why the destructors in LLVM are not always 
prepared to be executed in exception situations. Yes, we build LLVM 
with exception support. In principle, we build it with the same options 
like the rest of our project. Actually, I could hardly imagine that 
we could handle OOM situations without error handling. 
> For the particular cases you mentioned with auto pointers and allocation 
> in destructors, are these issues also present along existing error 
> paths?  Or for that matter simply examples of bad coding practice?  If 
> so, pushing back selected changes would be welcomed.  I'd be happy to 
> help review.
Yes, there are some examples of bad coding practice. The root problem,
however, is that the destructors in LLVM do a lot of complicated stuff. 
Instead of just deleting objects following a strictly hierarchical ownership 
structure, the objects are unregistered in various relationships, which 
potentially trigger unwinding in quite different locations. Such non-trivial 
coding sometimes requires dynamic allocation of new collections, which is
problematic in OOM situations. Actually, we did not manage to completely
fix the unwinding of the compiler state. That was one of the reasons to 
move all compiler passes to a separate process.

Here is a rough overview of our current set of patches to LLVM3.3. In total, 
we have 31 patches to LLVM related to OOM handing. They fix only the 
components that we could not outsource to the separate process: 
the core IR classes that we use for IR generation, IR bitcode serialization,
and the dynamic code loader. 

* In 8 of the patches we fix the malloc calls to throw bad_alloc.
* 10 patches fix destructors. Some of fixes are about disabling or
rewriting the code triggering dynamic allocations. Some other fixes 
disable asserts that check for invariants that are not valid in exception 
situation.
* 5  patches deal with exceptions in constructors. One kind of problems 
results from the fact that if an exception is thrown in the constructor of 
an object, the destructor is not called which causes a leak.
There is also a specific problem that IR objects register themselves with 
their parents already in their constructor. And if a constructor fails 
afterwards, the parent contains a dangling pointer to its child.
* 5 patches fix temporary ownership of objects, mostly the situations 
when object are created, but not added to the owning collections.

Note that 31 patches does not mean 31 fixes, because some of them 
contain all fixes to a particular file or class. 

Regards,
Vaidas