[Rails] Re: Database Concurrency Without a Web Server?

Isak Hansen
Thu, 13 Jul 2006 17:22:25 -0000

On 7/13/06, Don Stocks <amillionhitpoints@yahoo.com>
wrote:
> Francis Cianfrocca wrote:
> > Don, would it help if we added something like this to the
> > EventMachine...
> >
> > You know how Python''s Twisted has something called a
"deferred"?
> > Basically an operation you kick off and you supply a callback that
gets
>
> Guys, a huge "Thank You" for all the help!
>
> Francis, if I understand everything correctly, I think that would do the
> trick!  The db client blocking certainly explains why 100 monitors per
> minute runs like clockwork but 2000 introduces a bunch of delays.
>
> Right now, the only call to the db that I make within EventMachine is to
> save the record.
>
> Here is how it looks:
>
>   EventMachine.add_timer(delay){job.run; job.save}
>
> With the exception of job.save(), all other db activity happens before I
> add the job to EventMachine.  And I don''t think there is any
blocking
> code in job.run.  It simply executes my network connectivity code and
> sets the Job object''s attributes based on the results.
>
> As far as instantiating a bunch of objects, I think that''s working
just
> fine.  I can instantiate 1000 Job objects and add them to EventMachine
> in less than a second.  So I''m very happy there.
>
> If I could get job.save to execute in a thread pool I think I''ll
be
> cooking with gas.

I''m not sure introducing concurrency will boost your insert rate all
that much.


I whipped up a small test to check, but even with
AR::Base.allow_concurrency set to true it seems my db access is still
serialized. 10 connections are opened all right, but 9 are idle
whenever i check db stats.

Any thoughts? - Source at <http://pastebin.ca/87122>


Isak