llvm dev - Jun 2015 - [LLVMdev] "LLVM as a library" and signaling failures other than crashing

Hi all,

LLVM, in the absence of exception, does not have currently any way of signaling
error other than crashing (the client can register a handler but abort() is
always called anyway.

For a user trying to use LLVM “as a library” (for JIT for instance), it seems a
bit hostile to me to have to fork a process every time I want to make use of a
library.
Of course some errors might not be recoverable if the internal data structure
are corrupted, but that should be the exception. Having the register allocator
running out of registers/spill space should not need to tear down the whole
process.
In general, I’d rather leave up to the client the decision on what to do on
error. For instance the JIT javascript client can discard the compilation
failure.

Since a new PassManager is being brought up (CC Chandler), I figured it might be
a good time to raise this concern since the PassManager sits in the middle.
A potential solution would be to have the ability for passes to return a
“status” when run(). The PassManager can then early exit from the pipeline on
errors from a Pass and propagate the status to the client as well.

To bring this up, Passes can be changed to always return a status “success”,
which would match the existing behavior. We could then progressively wire places
where report_fatal_error() is called to return an error instead.

Any thoughts/interest in evolving this aspect of LLVM?

Best,

— 
Mehdi

On 06/04/2015 02:58 PM, Mehdi Amini wrote:
> Hi all,
>
> LLVM, in the absence of exception, does not have currently any way of
signaling error other than crashing (the client can register a handler but
abort() is always called anyway.
>
> For a user trying to use LLVM “as a library” (for JIT for instance), it
seems a bit hostile to me to have to fork a process every time I want to make
use of a library.
> Of course some errors might not be recoverable if the internal data
structure are corrupted, but that should be the exception. Having the register
allocator running out of registers/spill space should not need to tear down the
whole process.
> In general, I’d rather leave up to the client the decision on what to do on
error. For instance the JIT javascript client can discard the compilation
failure.
>
> Since a new PassManager is being brought up (CC Chandler), I figured it
might be a good time to raise this concern since the PassManager sits in the
middle.
> A potential solution would be to have the ability for passes to return a
“status” when run(). The PassManager can then early exit from the pipeline on
errors from a Pass and propagate the status to the client as well.
>
> To bring this up, Passes can be changed to always return a status
“success”, which would match the existing behavior. We could then progressively
wire places where report_fatal_error() is called to return an error instead.
>
> Any thoughts/interest in evolving this aspect of LLVM?
Speaking as a heavy user of llvm as a library for an in memory compiler, 
I have little interest in this proposal.  If I'm generating valid IR, 
the compiler doesn't really have a reasonable failure mode.  Every case 
where I've seen an LLVM crash has been either a serious bug in LLVM or a 
bug in my code leveraging it.  I do *not* want silent failure.  I would 
rather see these crash so that get dealt with.

This would be different if LLVM had a reason for telling me that certain 
functions I generate it couldn't compile - like say, it exceeded a 
memory limit I set - but that doesn't exist today and doesn't seem to be
likely to happen in the near future.  If you can enumerate a set of 
reasons for a graceful failure - like a misconfiguration? - I'd be open 
to see what we could do about those specific cases.

Wrapping a process around the compilation thread so that compiler bugs 
don't bring down the underlying VM is a reasonable thing to do.  
Teaching the compiler how to gracefully return error codes for each and 
every unknown bug is not.  I've seen it tried with another compiler 
before, the result is not pretty.

Philip

Hi Philip,

Thanks for your input.
> On Jun 4, 2015, at 4:08 PM, Philip Reames <listmail at
philipreames.com> wrote:
> 
> 
> 
> On 06/04/2015 02:58 PM, Mehdi Amini wrote:
>> Hi all,
>> 
>> LLVM, in the absence of exception, does not have currently any way of
signaling error other than crashing (the client can register a handler but
abort() is always called anyway.
>> 
>> For a user trying to use LLVM “as a library” (for JIT for instance), it
seems a bit hostile to me to have to fork a process every time I want to make
use of a library.
>> Of course some errors might not be recoverable if the internal data
structure are corrupted, but that should be the exception. Having the register
allocator running out of registers/spill space should not need to tear down the
whole process.
>> In general, I’d rather leave up to the client the decision on what to
do on error. For instance the JIT javascript client can discard the compilation
failure.
>> 
>> Since a new PassManager is being brought up (CC Chandler), I figured it
might be a good time to raise this concern since the PassManager sits in the
middle.
>> A potential solution would be to have the ability for passes to return
a “status” when run(). The PassManager can then early exit from the pipeline on
errors from a Pass and propagate the status to the client as well.
>> 
>> To bring this up, Passes can be changed to always return a status
“success”, which would match the existing behavior. We could then progressively
wire places where report_fatal_error() is called to return an error instead.
>> 
>> Any thoughts/interest in evolving this aspect of LLVM?
> Speaking as a heavy user of llvm as a library for an in memory compiler, I
have little interest in this proposal.  If I'm generating valid IR, the
compiler doesn't really have a reasonable failure mode.  Every case where
I've seen an LLVM crash has been either a serious bug in LLVM or a bug in my
code leveraging it.  I do *not* want silent failure.
What do you mean by “silent”? I want to *signal and report error*, but it does
imply necessarily crashing.

Crashing is “fine" for testing, but it is not an acceptable option in
production. If there is a fall back (an interpreter for instance), you really
don’t want to crash the process.
>  I would rather see these crash so that get dealt with.
There is nothing that prevents you (as a library client) from crashing when an
error occurs, but that can be dealt with outside of LLVM. This is how I am used
to interact with almost any library.

> This would be different if LLVM had a reason for telling me that certain
functions I generate it couldn't compile - like say, it exceeded a memory
limit I set - but that doesn't exist today and doesn't seem to be likely
to happen in the near future.  If you can enumerate a set of reasons for a
graceful failure - like a misconfiguration? - I'd be open to see what we
could do about those specific cases.
We have multiple passes that operate under some constraints (I mentioned for
instance a limit on the number of registers and spill size, but we have other
resources that we deal with), I don’t really know enough about “in tree” use
cases.
Not being able to generate valid code in these conditions is frequently an input
error (the user wrote some code that we can’t fit on the target is common), but
not a compiler bug (or at least not a bug in LLVM).
I don’t see why it would be required to crash in such condition and what is
preventing LLVM from returning properly to the client saying “Sorry I can’t do
what you are asking me”.

> Wrapping a process around the compilation thread so that compiler bugs
don't bring down the underlying VM is a reasonable thing to do. Teaching the
compiler how to gracefully return error codes for each and every unknown bug is
not.  I've seen it tried with another compiler before, the result is not
pretty.
I don’t propose to have something like a different error code for every single
possible failure, but more at the level of a single boolean (or close to that).
The existing infrastructure already allows to register a handler to pass a
textual diagnostic to the client, so that he can log it.

Wrapping a process around LLVM means that you can’t really use LLVM as a library
(in the sense that we use other libraries), you have to build a separate
executable around LLVM and implement some serialization.

— 
Mehdi