Eventmachine talk - Jul 2007 - Eventmachine 0.8.0 released with epoll,

The Eventmachine team is pleased to announce the release of version 0.8.0,
available now on RubyForge as a tarball and as a gem.

Eventmachine is a framework for writing fast, scalable network-aware
programs (clients, servers, or both) in Ruby or C/C++, without requiring the
use of threads. EM is used today in a variety of production systems
including Web, mail, and LDAP/RADIUS servers, SNMP agents, and custom
protocol handlers. It''s also used in Kirk Haines''s Swiftiply,
a fast
clustering proxy for Rails and other web frameworks.

The 0.8.0 release adds automatic support for epoll, on Linux 2.6 kernels.
This enables Eventmachine programs to break through Ruby''s limit of
1024
file and socket descriptors per process. In performance tests designed to
simulate busy web servers, Eventmachine with epoll exhibits the behavior you
would expect: with very large numbers (>20,000) of quiet connections and a
few active ones, there is no noticeable degradation of responsiveness or
total throughput, compared to cases with only a few connections.

We expect the next major EM release to be called 0.9.0. It will add
Erlang-like features intended to make it simple for EM-based programs to
send messages to each other, or to other processes. A typical use case for
this capability is a "service" web site that must make calls to other
web
sites, web services, or other network services in order to fulfill requests
from its own clients.

Thanks to all for your support of the EM project, and especially to Tony
Arcieri, who tirelessly bugged me for weeks to add the epoll support, and to
Kirk for helping to shake it out.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070711/373e391b/attachment.html

Hi Francis,

Any work being done on non blocking database drivers for EM?  I know
that''s been a major reason why I haven''t used EM in more
applications.
  The message passing stuff sounds cool though, can''t wait to see it.

Chris

Hi Francis,

Now that you have epoll support working (congrats BTW!), is kqueue
(FreeBSD/Mac OS X) or /dev/poll (Solaris) support far off the horizon?

Best regards,

--Michael

On 7/11/07, snacktime <snacktime at gmail.com>
wrote:
> Hi Francis,
>
> Any work being done on non blocking database drivers for EM?  I know
> that''s been a major reason why I haven''t used EM in more
applications.
>   The message passing stuff sounds cool though, can''t wait to see
it.
>
> Chris
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>
>

On 7/11/07, Michael S. Fischer <michael at dynamine.net>
wrote:
>
> Hi Francis,
>
> Now that you have epoll support working (congrats BTW!), is kqueue
> (FreeBSD/Mac OS X) or /dev/poll (Solaris) support far off the horizon?


A higher priority would probably be Windows/IOCP, to tell the truth.
I''m
completely amenable to adjusting the priorities, though, especially if
someone would like to step up and help develop and test, hint hint!. kqueue
and /dev/poll will be pretty easy for someone to fit into the epoll pattern.
IOCP is a complete rewrite.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070711/b46a20fb/attachment.html

On 7/11/07, snacktime <snacktime at gmail.com>
wrote:
>
> Hi Francis,
>
> Any work being done on non blocking database drivers for EM?  I know
> that''s been a major reason why I haven''t used EM in more
applications.
>   The message passing stuff sounds cool though, can''t wait to see
it.
>
> Chris


Chris, the "official" response to this is to run blocking DBMS calls
with
EventMachine#defer. That puts them on a thread pool managed by the EM
reactor.

I''d be thrilled if someone were to implement evented DB drivers,
though. You
could do it in Ruby with the existing protocol handlers in lib/protocols.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070711/c7be9c88/attachment.html

>
> Chris, the "official" response to this is to run blocking DBMS
calls with
> EventMachine#defer. That puts them on a thread pool managed by the EM
> reactor.
>
> I''d be thrilled if someone were to implement evented DB drivers,
though. You
> could do it in Ruby with the existing protocol handlers in lib/protocols.
>
I should have clarified.  I meant the ones written as C extensions.
Then again depending on the application you might not really be losing
much with a slower pure ruby driver.   More cpu time but no io
blocking, so it''s probably a win.

Chris
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>

On 7/11/07, snacktime <snacktime at gmail.com>
wrote:
>
> >
> > Chris, the "official" response to this is to run blocking
DBMS calls
> with
> > EventMachine#defer. That puts them on a thread pool managed by the EM
> > reactor.
> >
> > I''d be thrilled if someone were to implement evented DB
drivers, though.
> You
> > could do it in Ruby with the existing protocol handlers in
> lib/protocols.
> >
>
> I should have clarified.  I meant the ones written as C extensions.
> Then again depending on the application you might not really be losing
> much with a slower pure ruby driver.   More cpu time but no io
> blocking, so it''s probably a win.


But you''re still talking about changing  them so they use evented I/O,
right?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070711/d9bc1d29/attachment.html

>
> But you''re still talking about changing  them so they use evented
I/O,
> right?
>
Yes.  Not sure what you could gain by trying to do it in pure ruby
though, as you would still just have to use a thread right?  I think
the best option would be to just add it to existing C drivers where
the client library supports async io.  Postgres does, I was just
looking at their specs.

Chris

On 7/11/07, snacktime <snacktime at gmail.com>
wrote:
>
> >
>
> Yes.  Not sure what you could gain by trying to do it in pure ruby
> though, as you would still just have to use a thread right?  I think
> the best option would be to just add it to existing C drivers where
> the client library supports async io.  Postgres does, I was just
> looking at their specs.


The only reason I thought of doing it in Ruby is to save time, but your idea
makes a lot of sense too. Anyone out there want to give this a shot?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070711/49ca59b5/attachment-0001.html

On 7/11/07, Francis Cianfrocca <garbagecat10 at gmail.com>
wrote:
> On 7/11/07, snacktime <snacktime at gmail.com> wrote:
> > >
> >
> > Yes.  Not sure what you could gain by trying to do it in pure ruby
> > though, as you would still just have to use a thread right?  I think
> > the best option would be to just add it to existing C drivers where
> > the client library supports async io.  Postgres does, I was just
> > looking at their specs.
>
>
> The only reason I thought of doing it in Ruby is to save time, but your
idea
> makes a lot of sense too. Anyone out there want to give this a shot?
Interesting, there is already async support in the c driver I''ll have
to test it out.  It uses rb_thread_select to read the fd, so in theory
it should only block the thread that''s making the call to the driver,
not the whole process.  Wish I had looked at this a long time ago!

Chris