Eventmachine talk - Jul 2008 - Evented Database Access, Again

I have been playing with the PostgreSQL Async API (which the ruby-pg driver
happens to support) for a while.

The performance gain is impressive (up to 40% improvement over blocking
mode). You can read the details
here<http://oldmoe.blogspot.com/2008/07/faster-io-for-ruby-with-postgres.html>.


Now I am wondering if there is a way to integrate this tightly with EM.

I need to pass a socket (or a list of sockets) and a callback to EM. This
way I will be able to run an http server with one event loop for all the I/O
rather than the hack I am currently using (I am calling select for the pgsql
sockets in next_tick blocks)

regards

oldmoe
eSpace <http://www.espace.com.eg>
oldmoe.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/eventmachine-talk/attachments/20080712/10bb80d4/attachment.html>

It wouldn''t be too hard--you''d need to add a method to EM to
be able
to accept existing sockets and bind them within its own little TCP
class [and have that associated with a ruby class i.e. a member of
@sockets].

Rev supports adding raw sockets to event loops, I believe, though I''ve
never done one explicitly [I''ve always used the TCP Socket class].

This makes me wonder if the standard mysql libs have any asynchronous
ability.  I''ve never seen it.
Looks like it has quite a bit of potential.  Fascinating that nobody
has really used the PG drivers in an asynchronous way like this
before, despite it being possible.
Good luck!
-R

On Jul 11, 2008, at 7:56 PM, Muhammad A. Ali wrote:
> I have been playing with the PostgreSQL Async API (which the ruby-pg
> driver happens to support) for a while.
>
> The performance gain is impressive (up to 40% improvement over
> blocking mode). You can read the details here.
>
> Now I am wondering if there is a way to integrate this tightly with
> EM.
>
> I need to pass a socket (or a list of sockets) and a callback to EM.
> This way I will be able to run an http server with one event loop
> for all the I/O rather than the hack I am currently using (I am
> calling select for the pgsql sockets in next_tick blocks)
>
> regards
>
> oldmoe
> eSpace
> oldmoe.blogspot.com
>
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/eventmachine-talk/attachments/20080714/4ab3ddf4/attachment.html>

On 15 Jul 2008, at 02:15, Roger Pack wrote:
> It wouldn''t be too hard--you''d need to add a method to EM
to be able
> to accept existing sockets and bind them within its own little TCP
> class [and have that associated with a ruby class i.e. a member of
> @sockets].
I have taken a little stab at this, but I think I may have missed
something, I''ll catch up with Aman / others this week and see if I can
get some assistance on the C++ side.
> Rev supports adding raw sockets to event loops, I believe, though
> I''ve never done one explicitly [I''ve always used the TCP
Socket
> class].
>
> This makes me wonder if the standard mysql libs have any
> asynchronous ability.  I''ve never seen it.
> Looks like it has quite a bit of potential.  Fascinating that nobody
> has really used the PG drivers in an asynchronous way like this
> before, despite it being possible.
Oh they do, just not in the ruby world.

I had started a translation of postgres-pr, however using the native
library and attaching the reactor loop to avoid multiple solutions to
the I/O problem.

The one area that is slightly more complex here is that you need the
reactor api to not read from the socket as soon as the data is ready,
but instead to call consume input on the api.

A more generic solution would allow for general socket ready callbacks
irrespective of the specific reader implementation.
> Good luck!
> -R
>
> On Jul 11, 2008, at 7:56 PM, Muhammad A. Ali wrote:
>
>> I have been playing with the PostgreSQL Async API (which the ruby-
>> pg driver happens to support) for a while.
>>
>> The performance gain is impressive (up to 40% improvement over
>> blocking mode). You can read the details here.
>>
>> Now I am wondering if there is a way to integrate this tightly with
>> EM.
>>
>> I need to pass a socket (or a list of sockets) and a callback to
>> EM. This way I will be able to run an http server with one event
>> loop for all the I/O rather than the hack I am currently using (I
>> am calling select for the pgsql sockets in next_tick blocks)
>>
>> regards
>>
>> oldmoe
>> eSpace
>> oldmoe.blogspot.com
>>
>> _______________________________________________
>> Eventmachine-talk mailing list
>> Eventmachine-talk at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/eventmachine-talk/attachments/20080715/db341cf7/attachment-0001.html>