Rails - Jan 2007 - ActiveRecord on Threads: close and reopen the connection?

I''m writing a Rails application which does not have a web frontend  
(no controllers and no views, only models). Rather, it is a faceless  
server, supporting many tens of thousands of threads and socket  
connections. Some of these are fairly short-lived, perhaps a second  
or two at the most, but the majority of them live longer, possibly up  
to several hours or more.

Writing a server application in Rails has its advantages - among them  
is the testing framework, and of course the abstractions provided by  
ActiveRecord.

However, it is impractical to allot a database connection per thread,  
since there are so many of them. And most of the work done in a  
thread involves manipulation of ActiveRecord objects only initially,  
at the start of a thread''s life, and perhaps a couple of times  
thereafter.

For this application, it would make sense to close the db connection  
and reopen it when needed.

My question to those of you who know about the inner workings of  
ActiveRecord is: what is the best way of disconnecting from the  
database? Of course, I can wrap all ActiveRecord accesses in blocks,  
like this:

	with_connection do { MyModel.find(:all).each ... }

where with_connection could be defined as

	def with_connection
	  begin
	    ActiveRecord::Base.establish_connection
	    result = yield
	  ensure
	    ActiveRecord::Base.remove_connection
	  end
	  result
	end

With_connection could of course also maintain a pool of connections.

But is there a configuration alternative that automatically makes  
ActiveRecord drop the connection after each query?

I should add that ActiveRecord::Base.allow_concurrency = true, and  
that the db is Postgres, with concurrency enabled in the database.yml  
file.

	/ Peter Bengtson



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Hi~

On Jan 31, 2007, at 10:24 AM, Peter Bengtson wrote:
>
> I''m writing a Rails application which does not have a web frontend
> (no controllers and no views, only models). Rather, it is a faceless
> server, supporting many tens of thousands of threads and socket
> connections. Some of these are fairly short-lived, perhaps a second
> or two at the most, but the majority of them live longer, possibly up
> to several hours or more.
>
> Writing a server application in Rails has its advantages - among them
> is the testing framework, and of course the abstractions provided by
> ActiveRecord.
>
> However, it is impractical to allot a database connection per thread,
> since there are so many of them. And most of the work done in a
> thread involves manipulation of ActiveRecord objects only initially,
> at the start of a thread''s life, and perhaps a couple of times
> thereafter.
>
> For this application, it would make sense to close the db connection
> and reopen it when needed.
>
> My question to those of you who know about the inner workings of
> ActiveRecord is: what is the best way of disconnecting from the
> database? Of course, I can wrap all ActiveRecord accesses in blocks,
> like this:
>
> 	with_connection do { MyModel.find(:all).each ... }
>
> where with_connection could be defined as
>
> 	def with_connection
> 	  begin
> 	    ActiveRecord::Base.establish_connection
> 	    result = yield
> 	  ensure
> 	    ActiveRecord::Base.remove_connection
> 	  end
> 	  result
> 	end
>
> With_connection could of course also maintain a pool of connections.
>
> But is there a configuration alternative that automatically makes
> ActiveRecord drop the connection after each query?
>
> I should add that ActiveRecord::Base.allow_concurrency = true, and
> that the db is Postgres, with concurrency enabled in the database.yml
> file.
>
> 	/ Peter Bengtson

Peter-

	The above could work but in my experiments with using AR in threaded  
mode the easiest way to make sure connections close is to  
verify_active_connections. I assume you have a server.accept loop  
that is accepting new connections and firing up a thread for each  
one? AR will make a connection for each thread. But if you put a  
counter around your server.accept loop and then  
verfiy_active_connections! every 20-50 times through the loop it will  
reap old connections and keep things fairly clean.

hits = 0
while server.accept
   hits += 1
   Thread.new do
       # do your thang in here that calls AR
   end
   ActiveRecord::Base.verify_active_connections! if hits % 30
end

	That seems to work well when allow_concurrency is set to true. But  
in your case of needing thousands of sockets open then you may want  
to look at an even driven server like EvenMachine on rubyforge.


Cheers-
-- Ezra Zygmuntowicz 
-- Lead Rails Evangelist
-- ez-NLltGlunAUd/unjJdyJNww@public.gmane.org
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---