Eventmachine talk - Nov 2007 - Should I use deferrables?

Hi

I''m writing a simple server for our internal API, using
EventMachineHttpServer. The structure of the code is something like
this:

module MyServer
  include EventMachine::HttpServer
  def process_http_request
    # load the required classes and execute them,
    # meaning accessing SQL servers, IMAP servers, etc,
    # i.e., blocking calls.
  end
end

EM.epoll
EM.set_descriptor_table_size(65535)
EM.run do
  EM.start_server("0.0.0.0", 80, MyServer)
end

Using the this code (it''s actually somewhat more complex, because it
does some validations), I get around 550 req/s using apache''s
"ab"
tool[1].

If I add a "sleep(rand(2))" call the process_http_request method, it
results in a slow down, which eventually causes the benckmark to timeout
after around 250 reqs.

Would changing this code to use deferrables improve concurrency in this
situation? I''m guessing yes, since, quoting Francis, "[EM#defer
is]
indispensable for certain things (like making blocking calls to database
libraries)".

Does this abstraction work well with EM''s HttpServer? Would spawned
processes work better?

Examples of simple server code using deferrables or spawned processes
would be appreciated =)

Thanks in advance,
Andre

[1] I''m running "ab -c 1000 -n 10000 http://localhost/myurl"
for these
tests.

On Nov 19, 2007 10:34 AM, Andre Nathan <andre at digirati.com.br>
wrote:
> Hi
>
> I''m writing a simple server for our internal API, using
> EventMachineHttpServer. The structure of the code is something like
> this:

Short answer: yes.

Slightly longer: Deferrables and Spawned Processes are essentially
equivalent in that neither provides functionality not provided by the
other. It''s a matter of taste which one you use.

Look at the Unicycle RESTful framework for an example of how to use
Deferrable with the HTTP server. It works extremely well and is very
graceful and easy to use.

Hi Francis

On Mon, 2007-11-19 at 11:09 -0500, Francis Cianfrocca
wrote:
> Short answer: yes.
Does this look good for a simple skeleton code for a "deferrable
server"?

class MyClass
  include EventMachine::Deferrable

  def my_blocking_method(x)
    "my_blocking_method got #{x}"
  end
end

module MyServer
  include EventMachine::HttpServer

  def process_http_request
    response = EventMachine::DelegatedHttpResponse.new(self)
    fun = lambda do |status, result|
      response.status = status
      response.content = result
      response.send_response
    end
    c = MyClass.new
    c.callback { |result| fun[200, result] }
    c.errback { |result| fun[500, result] }
    begin
      result = c.my_blocking_method("foo")
      c.succeed result.to_s
    rescue Exception => e
      c.fail e.to_s
    ensure
      close_connection_after_writing
    end
  end
end

EM.run do
  EM.start_server("0.0.0.0", 8080, MyServer)
end

On Nov 19, 2007 12:18 PM, Andre Nathan <andre at digirati.com.br> wrote:
 > Does this look good for a simple skeleton code for a
"deferrable
> server"?
>
> class MyClass
>   include EventMachine::Deferrable
>
>   def my_blocking_method(x)
>     "my_blocking_method got #{x}"
>   end
> end
>
> module MyServer
>   include EventMachine::HttpServer
>
>   def process_http_request
>     response = EventMachine::DelegatedHttpResponse.new(self)
>     fun = lambda do |status, result|
>       response.status = status
>       response.content = result
>       response.send_response
>     end
>     c = MyClass.new
>     c.callback { |result| fun[200, result] }
>     c.errback { |result| fun[500, result] }
>     begin
>       result = c.my_blocking_method("foo")
>       c.succeed result.to_s
>     rescue Exception => e
>       c.fail e.to_s
>     ensure
>       close_connection_after_writing
>     end
>   end
> end
>
> EM.run do
>   EM.start_server("0.0.0.0", 8080, MyServer)
> end
>
>
Yes, that''s the idea, although you can probably refactor it somewhat
to make it more graceful. Did you look at protocol.rb in Unicycle?

On Mon, 2007-11-19 at 12:24 -0500, Francis Cianfrocca
wrote:
> Yes, that''s the idea, although you can probably refactor it
somewhat
> to make it more graceful. Did you look at protocol.rb in Unicycle?
Yeah, I looked there. I just didn''t want to make the example too
complicated.

I''ll change my code to use deferrables then, and resport the results.

Thanks!

Andre

On Nov 19, 2007 12:26 PM, Andre Nathan <andre at digirati.com.br>
wrote:
> On Mon, 2007-11-19 at 12:24 -0500, Francis Cianfrocca wrote:
> > Yes, that''s the idea, although you can probably refactor it
somewhat
> > to make it more graceful. Did you look at protocol.rb in Unicycle?
>
> Yeah, I looked there. I just didn''t want to make the example too
> complicated.
>
> I''ll change my code to use deferrables then, and resport the
results.

Looking forward to your report. I''ve found Deferrables to be hard to
understand at first, but incredibly graceful once you''re used to them.
And it''s true that Python''s Twisted uses their
"deferred" object
pervasively, so there must be something to it.

For those who are interested in learning more, there''s a document
called DEFERRABLES in the EM distro.

On Mon, 2007-11-19 at 12:32 -0500, Francis Cianfrocca
wrote:
> Looking forward to your report. I''ve found Deferrables to be hard
to
> understand at first, but incredibly graceful once you''re used to
them.
> And it''s true that Python''s Twisted uses their
"deferred" object
> pervasively, so there must be something to it.
>
> For those who are interested in learning more, there''s a document
> called DEFERRABLES in the EM distro.
Do you think you could add that little server skeleton to that document?
I''ve read it, but the whole thing only became clearer to me after
reading unicycle''s code.

Andre

On Nov 19, 2007 1:03 PM, Andre Nathan <andre at digirati.com.br>
wrote:
> On Mon, 2007-11-19 at 12:32 -0500, Francis Cianfrocca wrote:
> > Looking forward to your report. I''ve found Deferrables to be
hard to
> > understand at first, but incredibly graceful once you''re used
to them.
> > And it''s true that Python''s Twisted uses their
"deferred" object
> > pervasively, so there must be something to it.
> >
> > For those who are interested in learning more, there''s a
document
> > called DEFERRABLES in the EM distro.
>
> Do you think you could add that little server skeleton to that document?
> I''ve read it, but the whole thing only became clearer to me after
> reading unicycle''s code.

Good idea.

On Mon, 2007-11-19 at 12:24 -0500, Francis Cianfrocca
wrote:
> Yes, that''s the idea, although you can probably refactor it
somewhat
> to make it more graceful. Did you look at protocol.rb in Unicycle?
I''ve added a "sleep(rand(2))" call before returning the
string in
#my_blocking_method, to simulate a blocking call, and I''m seeing the
same behaviour I had before I used deferrables (i.e., timeouts when
trying to access the server from the benchmark tool).

Am I just using a bad way to simulate the blocking calls or is this
unexpected?

Thanks,
Andre

On Nov 19, 2007 5:33 PM, Andre Nathan <andre at digirati.com.br>
wrote:
> On Mon, 2007-11-19 at 12:24 -0500, Francis Cianfrocca wrote:
> > Yes, that''s the idea, although you can probably refactor it
somewhat
> > to make it more graceful. Did you look at protocol.rb in Unicycle?
>
> I''ve added a "sleep(rand(2))" call before returning the
string in
> #my_blocking_method, to simulate a blocking call, and I''m seeing
the
> same behaviour I had before I used deferrables (i.e., timeouts when
> trying to access the server from the benchmark tool).
>
> Am I just using a bad way to simulate the blocking calls or is this
> unexpected?

If you really want to simulate a blocking call from within EM, use #defer:

EM.defer {sleep( rand( 2 ))}

That will run the sleep on a Ruby thread managed by the reactor.

You would do the same thing with a blocking call to a DB library.

However, if you were making a call to a secondary web site, then you''d
want to use EM''s HTTP client, which itself returns a Deferrable, and
you get the full non-blocking magic.

On Mon, 2007-11-19 at 17:40 -0500, Francis Cianfrocca
wrote:
> If you really want to simulate a blocking call from within EM, use #defer:
>
> EM.defer {sleep( rand( 2 ))}
>
> That will run the sleep on a Ruby thread managed by the reactor.
>
> You would do the same thing with a blocking call to a DB library.
>
> However, if you were making a call to a secondary web site, then
you''d
> want to use EM''s HTTP client, which itself returns a Deferrable,
and
> you get the full non-blocking magic.
No, the blocking calls will be database queries 99% of the time. Using
EM.defer I get the same benckmark results as the ones i get without the
call (even with "EM.defer proc { sleep(60) }" or some other large
value), that is, it''s not affecting the number of req/s. Is that what I
should expect to see? I was expecting it to get at least a bit worse :)

Just to make sure I understood how this is working: when I include
Deferrable in my class, does that mean that every method called on an
object of that class will be automatically deferred, or do I still need
to manually wrap long-running method calls on an "EM.defer(proc {...},
proc {...})" pair of blocks?

My users will write their own classes that the server will load, and I
want to expose as little as possible of how the server works to them, so
ideally they''d write their code as usual (do db connections, return a
hash with the results). The classes they write must inherit from a class
that includes Deferrable, so I''m assuming that it''ll work
transparently
for them.

Thanks,
Andre

On Nov 19, 2007 8:41 PM, Andre Nathan <andre at digirati.com.br>
wrote:
> On Mon, 2007-11-19 at 17:40 -0500, Francis Cianfrocca wrote:
> > If you really want to simulate a blocking call from within EM, use
#defer:
> >
> > EM.defer {sleep( rand( 2 ))}
> >
> > That will run the sleep on a Ruby thread managed by the reactor.
> >
> > You would do the same thing with a blocking call to a DB library.
> >
> > However, if you were making a call to a secondary web site, then
you''d
> > want to use EM''s HTTP client, which itself returns a
Deferrable, and
> > you get the full non-blocking magic.
>
> No, the blocking calls will be database queries 99% of the time. Using
> EM.defer I get the same benckmark results as the ones i get without the
> call (even with "EM.defer proc { sleep(60) }" or some other large
> value), that is, it''s not affecting the number of req/s. Is that
what I
> should expect to see? I was expecting it to get at least a bit worse :)
>
> Just to make sure I understood how this is working: when I include
> Deferrable in my class, does that mean that every method called on an
> object of that class will be automatically deferred, or do I still need
> to manually wrap long-running method calls on an "EM.defer(proc {...},
> proc {...})" pair of blocks?
>
> My users will write their own classes that the server will load, and I
> want to expose as little as possible of how the server works to them, so
> ideally they''d write their code as usual (do db connections,
return a
> hash with the results). The classes they write must inherit from a class
> that includes Deferrable, so I''m assuming that it''ll work
transparently
> for them.
>
Despite the names, Deferrable and #defer have nothing to do with each other.

I assume your users will be writing code that you will invoke in
response to HTTP requests, ans that the user-written code will usually
make DB calls.

In that case you''re stuck using both #defer and Deferrable.
I''d
suggest something like this:

def your_http_processor
  d = instantiate_some_user_written_deferrable

  EM.defer {
    call_the_user_defined_entry_point
  }, {|result|
    d.callback result
  }

  d
end

What''s happening here is that you use #defer to call your
users'' code
on a separate thread (you need to warn your users that their code must
be threadsafe). When each call to a user-written function returns, the
thread will end and the main reactor thread will then call d.callback,
passing the result of the user function to the deferrable. At that
point, the HTTP client will get a response.

Using deferrable in this way is extremely fast and scalable.
Unfortunately, the use of #defer introduces a performance and
scalability bottleneck because of the Ruby threads.

If you really need to scale this up in a bigger way, you might
abstract your database calls out of the HTTP process.

On Nov 19, 2007 8:41 PM, Andre Nathan <andre at digirati.com.br> wrote:
 > Just to make sure I understood how this is working: when I
include
> Deferrable in my class, does that mean that every method called on an
> object of that class will be automatically deferred, or do I still need
> to manually wrap long-running method calls on an "EM.defer(proc {...},
> proc {...})" pair of blocks?
>
The only thing that including Deferrable does to a class is to add the
callback, errback, timeout, and set_deferred_status methods to it.
(set_deferred_status has several sugarings, like succeed and fail).

If you don''t call set_deferred_status on a Deferrable object, nothing
will happen. Deferrable is quite safe to include into arbitrary
objects.

If you want a Deferrable object without including the module in one of
your classes, instantiate the class EM::DefaultDeferrable.

On Mon, 2007-11-19 at 20:55 -0500, Francis Cianfrocca
wrote:
> Using deferrable in this way is extremely fast and scalable.
> Unfortunately, the use of #defer introduces a performance and
> scalability bottleneck because of the Ruby threads.
OK, this seems to work (for some reason now I have to kill -9 the server
to stop it, but it''s kinda late to look into that now):

  def process_http_request
    c = MyClass.new
    operation = proc do
      begin
        res = c.my_blocking_method("foo")
        [200, res]
      rescue Exception => e
        [500, e.to_s]
      end
    end
    callback = proc do |res|
      resp = EventMachine::DelegatedHttpResponse.new(self)
      resp.status = res[0]
      resp.content = res[1]
      resp.send_response
      resp.close_connection_after_writing
    end
    EM.defer(operation, callback)
  end

Error handling in the "operation" proc is a ugly but I
couldn''t think of
a better way to do it for now...

Do you have any plans for multi-process concurrency support in EM in the
future? I think it''s kinda risky for me to rely on ruby threads because
I don''t know in advance what kind of thing people will need to do in
their code.

Andre

On Nov 19, 2007 10:55 PM, Andre Nathan <andre at digirati.com.br> wrote:
 >
> Do you have any plans for multi-process concurrency support in EM in the
> future? I think it''s kinda risky for me to rely on ruby threads
because
> I don''t know in advance what kind of thing people will need to do
in
> their code.
>
I very much want to support multi-process concurrency. I think this
will be a very effective way to take advantage of multicore hardware.
Everyone assumes that threaded programming is the answer to this
issue, but we''ve all learned the hard way just how painful threads can
be.

If you''ve looked at EM''s spawnable processes, the intention is
these
will work across OS processes (and of course across machines) in the
same way that Erlang''s spawned processes do.

I don''t really care for the rather old-fashioned way that Erlang does
it (it relies on portmapper so it doesn''t play nice in the sandbox
with firewalls), but I''m sure we''ll come up with a better way.
Possibly with DNS service records.

Spawnable processes are more appropriate for cross-process
functionality than Deferrables, because they naturally support the
concept of idempotent message passing, which Deferrables don''t do.

Jruby gives multi-process support, but it looks that (at least for
now) 1.9 is going to be ''native thread one at a time'' (only
does one
thread at a time), so maybe look at jruby for now.
Cheers.
-Roger

On Nov 19, 2007 9:49 PM, Francis Cianfrocca <garbagecat10 at gmail.com>
wrote:
> On Nov 19, 2007 10:55 PM, Andre Nathan <andre at digirati.com.br>
wrote:
>  >
> > Do you have any plans for multi-process concurrency support in EM in
the
> > future? I think it''s kinda risky for me to rely on ruby
threads because
> > I don''t know in advance what kind of thing people will need
to do in
> > their code.
> >
>
> I very much want to support multi-process concurrency. I think this
> will be a very effective way to take advantage of multicore hardware.
> Everyone assumes that threaded programming is the answer to this
> issue, but we''ve all learned the hard way just how painful threads
can
> be.
>
> If you''ve looked at EM''s spawnable processes, the
intention is these
> will work across OS processes (and of course across machines) in the
> same way that Erlang''s spawned processes do.
>
> I don''t really care for the rather old-fashioned way that Erlang
does
> it (it relies on portmapper so it doesn''t play nice in the sandbox
> with firewalls), but I''m sure we''ll come up with a better
way.
> Possibly with DNS service records.
>
> Spawnable processes are more appropriate for cross-process
> functionality than Deferrables, because they naturally support the
> concept of idempotent message passing, which Deferrables don''t do.
>
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>


--
-Roger Pack
For God hath not given us the spirit of fear; but of power, and of
love, and of a sound mind" -- 2 Timothy 1:7

On Nov 19, 2007 11:58 PM, Roger Pack <rogerpack2005 at gmail.com>
wrote:
> Jruby gives multi-process support, but it looks that (at least for
> now) 1.9 is going to be ''native thread one at a time''
(only does one
> thread at a time), so maybe look at jruby for now.
> Cheers.
> -Roger

The Java version of EM works quite well, but it doesn''t remove the
basic hairball of threaded programming. I still think a multi-process
concurrency model is required in order to achieve both high
performance and robust programming.

On Nov 19, 2007 9:58 PM, Roger Pack <rogerpack2005 at gmail.com>
wrote:
> Jruby gives multi-process support, but it looks that (at least for
> now) 1.9 is going to be ''native thread one at a time''
(only does one
> thread at a time), so maybe look at jruby for now.
jruby has it''s Ruby threading model built on top of Java threads, for
both better and worse.

The better is that the threads are what many people expect from
threads -- actual native OS threads that are scheduled separately from
the parent.

The worse is that this means that they are much more heavyweight than
Ruby green threads, which means that things which can spawn a lot of
threads in short succession (such as threaded Mongrel) experience a
performance hit because of the heavier nature of the native threads,
requiring architecture changes to deal with the differences (thread
pools in the Java world).

I think that Francis is correct in that event based processes in a
multi-process architecture, with a nice message passing system between
them, is where the sweet spot lies between performance and ease of
implementation.


Kirk Haines

Is there an evented jruby mongrel?
Just wondering.
Thanks Kirk.
-Roger

On Nov 20, 2007 8:52 AM, Roger Pack <rogerpack2005 at gmail.com>
wrote:
> Is there an evented jruby mongrel?
> Just wondering.
> Thanks Kirk.
I just haven''t had enough time to even look at what needs to happen to
enable this in jmongrel.  I''m still looking for a poorly paid intern
to help with HTML/CSS wrangling and light programming in order to free
up some of my cycles so I can look at things like this.  :)


Kirk Haines