Eventmachine talk - Feb 2007 - Crash occurs where EventMachine.connect is

Hi everyone,

If you subclass EventMachine::Connection and call it outside an
EventMachine::run
event loop, EventMachine crashes!  Example:

class Put < EventMachine::Connection
   include EventMachine::Deferrable

   HOST="localhost"
   PORT=8080

   def self.request(data)
     EventMachine.connect(HOST, PORT, self) {|c|
       c.instance_eval { @data = data }
     }
   end

   # [...]
end

x = Put.request(data)

Crash:
terminate called after throwing an instance of
''std::runtime_error''
 what():  not initialized

Program received signal SIGABRT, Aborted.
[Switching to Thread -1211168096 (LWP 8580)]
0xffffe410 in __kernel_vsyscall ()
(gdb) bt
#0  0xffffe410 in __kernel_vsyscall ()
#1  0xb7d1a770 in raise () from /lib/tls/i686/cmov/libc.so.6
#2  0xb7d1bef3 in abort () from /lib/tls/i686/cmov/libc.so.6
#3  0xb7b34520 in __gnu_cxx::__verbose_terminate_handler ()
   at ../../../../src/libstdc++-v3/libsupc++/vterminate.cc:97
#4  0xb7b31f55 in __cxxabiv1::__terminate (
   handler=0xb7b343d0 <__gnu_cxx::__verbose_terminate_handler()>)
   at ../../../../src/libstdc++-v3/libsupc++/eh_terminate.cc:43
#5  0xb7b31f92 in std::terminate ()
   at ../../../../src/libstdc++-v3/libsupc++/eh_terminate.cc:53
#6  0xb7b320ca in __cxa_throw (obj=0x8117238, tinfo=0xb7b5531c,
   dest=0xb7ad0d50 <~runtime_error>)
   at ../../../../src/libstdc++-v3/libsupc++/eh_throw.cc:77
#7  0xb7b6e017 in evma_connect_to_server (server=0x80ccad8
"localhost",
   port=8080) at cmain.cpp:92
#8  0xb7b6c455 in t_connect_server (self=3082803140, server=3082609960,
   port=16161) at rubymain.cpp:207

Is it possible to die more gracefully, i.e. by raising an exception?

Alternatively, might there be some way to create a class that provides
a singleton method (e.g. self.connect) to return an instance of
EventMachine::connection running inside an anonymous event loop?
(Probably not, but I''m just curious.)

Best regards,

--Michael

On 2/26/07, Michael S. Fischer <michael at dynamine.net>
wrote:
>
> Hi everyone,
>
> If you subclass EventMachine::Connection and call it outside an
> EventMachine::run
> event loop, EventMachine crashes!  Example:
>
> class Put < EventMachine::Connection
>    include EventMachine::Deferrable
>
>    HOST="localhost"
>    PORT=8080
>
>    def self.request(data)
>      EventMachine.connect(HOST, PORT, self) {|c|
>        c.instance_eval { @data = data }
>      }
>    end
>
>    # [...]
> end
>
> x = Put.request(data)


Michael, this is actually the defined behavior. Most of EventMachine''s
functionality may only be invoked while EM''s "reactor" is
running (usually,
inside the block passed to EventMachine#run).


If you look inside ext/cmain.cpp, you''ll see where the exceptions are
being
thrown. For example, the function evma_connect_to_server (called by
EventMachine#connect) looks like this:


extern "C" const char *evma_connect_to_server (const char *server, int
port)
{
        if (!EventMachine)
                throw std::runtime_error ("not initialized");
        return EventMachine->ConnectToServer (server, port);
}



This is a little problematical in regard to throwing a Ruby exception
instead. EM was designed to be completely independent of Ruby (that turned
out not to be 100% possible, but it''s close). So the code that
interfaces EM
to Ruby is one level higher than this code (in ext/rubymain.cpp). Having
said that, I completely agree that throwing a C++ exception isn''t a
very
graceful way to
terminate a Ruby program! It might be possible (and easy) to put a C++ event
handler in rubymain.cpp, that would raise a Ruby exception. If anyone wants
to paw through the code and suggest a patch, I''m willing to consider
it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070226/81d1ae46/attachment.html

On 2/26/07, Michael S. Fischer <michael at dynamine.net>
wrote:
>
> Hi everyone,
>
> If you subclass EventMachine::Connection and call it outside an
> EventMachine::run
> event loop, EventMachine crashes!  Example:


Ok, I tried it. ext/rubymain.cpp contains the standard Ruby library wrapper,
which in EM''s case functions to massage data to and from Rubyland data
objects. I replaced t_connect_server with this:

static VALUE t_connect_server (VALUE self, VALUE server, VALUE port)
{
        try {
          const char *f = evma_connect_to_server (StringValuePtr(server),
NUM2INT(port));
          if (!f || !*f)
                  rb_raise (rb_eRuntimeError, "no connection");
          return rb_str_new2 (f);
        } catch (std::runtime_error &e) {
                 rb_raise (rb_eRuntimeError, e.what());
        }
}


and it works. This program:

require ''rubygems''
require ''eventmachine''

EventMachine.connect "127.0.0.1", 8000

now dies with a Ruby exception rather than a C++ exception. I''m not
convinced the extra exception-friendliness is worth the performance hit of
adding a C++ try/catch to every EM library function, especially since
it''s a
serious logic error to call an EM function while the reactor is running, and
I''m not sure it should be recoverable. Any better ideas out there?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070226/bcc2b726/attachment.html

On 2/26/07, Francis Cianfrocca <garbagecat10 at gmail.com>
wrote:
>
>  now dies with a Ruby exception rather than a C++ exception. I''m
not
> convinced the extra exception-friendliness is worth the performance hit of
> adding a C++ try/catch to every EM library function, especially since
it''s a
> serious logic error to call an EM function while the reactor is running,
and
> I''m not sure it should be recoverable. Any better ideas out there?
>

I meant "while the reactor is NOT running"
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070226/0caf770e/attachment.html

On 2/26/07, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:
> now dies with a Ruby exception rather than a C++ exception. I''m
not
> convinced the extra exception-friendliness is worth the performance hit of
> adding a C++ try/catch to every EM library function, especially since
it''s a
> serious logic error to call an EM function while the reactor is [not]
running, and
> I''m not sure it should be recoverable. Any better ideas out there?
Can you quantify the performance penalty incurred by performing the
check?  i.e., if it''s less than a few tenths of a percent in terms of
connections per second, is it worth worrying about?  Some points of
note:

1) I would guess (though I cannot be certain) that developers will
often implement exception handling in their Ruby code anyway, so
they''re often willing to take the small performance hit for the sake
of graceful error recovery;

2) EventMachine is not an island - when used as a component (for
example, in a Rails app), developers may be stymied when their entire
application crashes unexpectedly.

At the very least, it might be helpful to annotate the error message
so that users receive feedback more helpful in tracking down the
problem than "terminate called after throwing an instance of
''std::runtime_error''  what():  not initialized."  This
message tells
the developer or system administrator next to nothing about what
failed and why (terminate called by whom?  what was not initialized?);
I had to break out gdb to figure out what was going on, which is a
task beyond the ken of many Ruby developers and junior-to-intermediate
sysadmins.

Even a succinct but explanatory error message such as "connect called
outside EventMachine loop" would be better than what is there now.

Best regards,

--Michael

On 2/26/07, Michael S. Fischer <michael at dynamine.net>
wrote:
>
> On 2/26/07, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:
>
> > now dies with a Ruby exception rather than a C++ exception.
I''m not
> > convinced the extra exception-friendliness is worth the performance
hit
> of
> > adding a C++ try/catch to every EM library function, especially since
> it''s a
> > serious logic error to call an EM function while the reactor is [not]
> running, and
> > I''m not sure it should be recoverable. Any better ideas out
there?
>
> Can you quantify the performance penalty incurred by performing the
> check?  i.e., if it''s less than a few tenths of a percent in terms
of
> connections per second, is it worth worrying about?  Some points of
> note:
>
> 1) I would guess (though I cannot be certain) that developers will
> often implement exception handling in their Ruby code anyway, so
> they''re often willing to take the small performance hit for the
sake
> of graceful error recovery;
>
> 2) EventMachine is not an island - when used as a component (for
> example, in a Rails app), developers may be stymied when their entire
> application crashes unexpectedly.
>
> At the very least, it might be helpful to annotate the error message
> so that users receive feedback more helpful in tracking down the
> problem than "terminate called after throwing an instance of
> ''std::runtime_error''  what():  not initialized." 
This message tells
> the developer or system administrator next to nothing about what
> failed and why (terminate called by whom?  what was not initialized?);
> I had to break out gdb to figure out what was going on, which is a
> task beyond the ken of many Ruby developers and junior-to-intermediate
> sysadmins.
>
> Even a succinct but explanatory error message such as "connect called
> outside EventMachine loop" would be better than what is there now.



These are good points. Are you handy with C++? If so, feel free to add the
exception code I suggested above (or a better implementation if you come up
with one) and profile it. try/catch throws a lot of stuff on and off the
runtime stack, and in a busy application that could easily happen thousands
of times a second.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070226/48be5ae0/attachment.html

On 2/26/07, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:
> These are good points. Are you handy with C++? If so, feel free to add the
> exception code I suggested above (or a better implementation if you come up
> with one) and profile it. try/catch throws a lot of stuff on and off the
> runtime stack, and in a busy application that could easily happen thousands
> of times a second.
I''m no good with C++, but I can patch the EventMachine source and
rebuild it.  Unfortunately, I am not yet in a position to profile any
such changes because the project I''m working on is still under
development.  I''m a long way off from doing any performance profiling.

For the time being, would you consider more verbose error messaging?

Best regards,

--Michael