Eventmachine talk - Jul 2007 - EM and a rails messaging plug-in (a13g)

Hi,

I''m poised (figuratively) to start using the Ruby''s
ActiveMessaging
(a13g) project / Rails plugin, and was a little surprised by the
performance compared to JMS and JRuby:
http://beechbonanza.blogspot.com/2007/07/performance-of-jruby-jms-messaging.html

I thought a13g might be able to exploit some of EM''s good work.
Unfortunately I''m not expert enough in either to say definitively or
how.
Anyway, I''ve posted some msgs to activemessaging-discuss pointing this
way.

I''ve read the EM/rails thread
(http://rubyforge.org/pipermail/eventmachine-talk/2007-March/000462.html)
and it seems it may not be a trivial issue to ''marry'' Rails
and EM....

A thread of the a13g discussion of performance is:
http://groups.google.com/group/activemessaging-discuss/browse_thread/thread/f091dd4e107412be/5e652b8e0fdafd18?lnk=gst&q=performance&rnum=1#5e652b8e0fdafd18

It sounds like (early days still) a13g might have two ''code
paths'' for
performance: one if you run under JRuby and another if you run under
MRI - multiple pollers being run in the MRI case.
So I noticed that a Java/JRuby implementation of EM was planned in the
0.7.2 release - is this still the case?

In case it helps any potential exchanges here is a brief description
of how a13g (currently) fits within rails (See the first post by
Andrew Kuklewicz):

http://groups.google.com/group/activemessaging-discuss/browse_thread/thread/7517b89622959d91/720d1701a1a60e88?lnk=gst&q=activemessaging+sylvain&rnum=2#720d1701a1a60e88

I''m still trying to clarify to what extent a13g will
''just'' use the
rails plug-in infrastructure after any changes are made to improve
performance.

Regards
Mark

I worked a bit on stompserver a while back when we needed a message
queue and ran into bugs with the stomp protocol in Activemq.  I put in
a file and dbm backend storage to replace madeleine (which blows up
fairly quickly) and was easily reaching around 3000 messages per
second sustained throughput with file storage and a tad less with dbm
if I remember correctly.  Sql storage was considerably slower,
probably due to a mixture of the sql overhead and the sqlite driver
blocking.  The bottlenecks were pretty much all in the IO, cpu
utilization was low.

Chris

On 7/12/07, snacktime <snacktime at gmail.com>
wrote:
> I worked a bit on stompserver a while back when we needed a message
> queue and ran into bugs with the stomp protocol in Activemq.  I put in
> a file and dbm backend storage to replace madeleine (which blows up
> fairly quickly) and was easily reaching around 3000 messages per
> second sustained throughput with file storage and a tad less with dbm
> if I remember correctly.  Sql storage was considerably slower,
> probably due to a mixture of the sql overhead and the sqlite driver
> blocking.  The bottlenecks were pretty much all in the IO, cpu
> utilization was low.
Given that Analogger will easily do 10-20 times that, depending on
message length and hardware, passing a message from a client up to the
server, I would think a person should be able to do better than
3k/second with EM for a full messaging system.  I''ve thought about
that sort of a project myself, but but there''s only so much non-paying
work I can fit into my weeks, and it sounds like Francis is working on
building right into EM some erlang-inspired messaging features which
might change the details about how one would implement a messaging
system on top of EM.


Kirk Haines

On 7/12/07, Kirk Haines <wyhaines at gmail.com>
wrote:
> On 7/12/07, snacktime <snacktime at gmail.com> wrote:
> > I worked a bit on stompserver a while back when we needed a message
> > queue and ran into bugs with the stomp protocol in Activemq.  I put in
> > a file and dbm backend storage to replace madeleine (which blows up
> > fairly quickly) and was easily reaching around 3000 messages per
> > second sustained throughput with file storage and a tad less with dbm
> > if I remember correctly.  Sql storage was considerably slower,
> > probably due to a mixture of the sql overhead and the sqlite driver
> > blocking.  The bottlenecks were pretty much all in the IO, cpu
> > utilization was low.
>
> Given that Analogger will easily do 10-20 times that, depending on
> message length and hardware, passing a message from a client up to the
> server, I would think a person should be able to do better than
> 3k/second with EM for a full messaging system.  I''ve thought about
> that sort of a project myself, but but there''s only so much
non-paying
> work I can fit into my weeks, and it sounds like Francis is working on
> building right into EM some erlang-inspired messaging features which
> might change the details about how one would implement a messaging
> system on top of EM.
>
Actually the file storage was up around 5k/second, dbm was the one
around 3k, and those numbers were for a single client.  I did some
stress testing with up to 3 clients I believe but I can''t remember the
numbers.  I''m sure there are a lot of things that could be optimized
more, but it was plenty fast for me at the time so I didn''t take it
any further.

I''m looking forward to seeing what Francis is cooking up.

Chris

On 7/12/07, Mark Van De Vyver <mvyver at gmail.com>
wrote:
>
>  I''m poised (figuratively) to start using the Ruby''s
ActiveMessaging
> (a13g) project / Rails plugin, and was a little surprised by the
> performance compared to JMS and JRuby:
>
>
http://beechbonanza.blogspot.com/2007/07/performance-of-jruby-jms-messaging.html



I''m still chewing through what''s been said on this very
interesting thread,
but for starters, I''ll clarify that a pure-Java version of EM is
DEFINITELY
on the near-term roadmap. I won''t predict what version number of EM it
will
appear in. But it will be a feature-complete and interface-equivalent
alternative implementation of the EM reactor core in Java.

The Java version and the C++ version will both be supported indefinitely and
in parallel.

The driver for a Jruby-capable EM is a large new project that will be
announced in a few days. It will be an XACML-compliant access-control engine
for externalized policy-management. So stay tuned for that announcement.

If there are any Java gurus out there who would like to help out, now''s
the
time to raise your hand! :-)
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070713/974d0b9c/attachment.html