Has anyone thought about or tried to do some sort of database connection pooling in the adapter classes? Some databases (Oracle) are pretty slow at opening connections, so a connection pool could help with Oracle rails apps for sure. Is this really a Hard Problem to tackle? -Corey Lawson
* Corey Lawson [2005-07-05 14:36]:> Has anyone thought about or tried to do some sort of database > connection pooling in the adapter classes? > > Some databases (Oracle) are pretty slow at opening connections, so a > connection pool could help with Oracle rails apps for sure. >I could totally be wrong on this, but I believe since rails runs in a process model there''s no need for connection pooling. As each process starts up it opens a DB connection - and then each process only handles one request at a time, no threading. So there''s never any contention or need for additional db handles. And most opinions seem to be that using FastCGI in static mode is the way to go, that is not having it/apache manage the number of FCGI procs dynamically, but rather starting N procs upon startup. So your slow Oracle connections would all be opened upon app startup (like a db conn pool) and you shouldn''t have any connection time issues from then on. -- ________________________________ toddgrimason*todd[ at ]slack.net
I still would like to hear how many fcgi procs a site like backpack uses, or any other high-traffic rails sites for that matter. -PJ On 7/5/05, Todd Grimason <todd-cwT7Wi5Y1r1eoWH0uzbU5w@public.gmane.org> wrote:> * Corey Lawson [2005-07-05 14:36]: > > Has anyone thought about or tried to do some sort of database > > connection pooling in the adapter classes? > > > > Some databases (Oracle) are pretty slow at opening connections, so a > > connection pool could help with Oracle rails apps for sure. > > > > I could totally be wrong on this, but I believe since rails runs in a > process model there''s no need for connection pooling. As each process > starts up it opens a DB connection - and then each process only handles > one request at a time, no threading. So there''s never any contention or > need for additional db handles. > > And most opinions seem to be that using FastCGI in static mode is the > way to go, that is not having it/apache manage the number of FCGI procs > dynamically, but rather starting N procs upon startup. So your slow > Oracle connections would all be opened upon app startup (like a db conn > pool) and you shouldn''t have any connection time issues from then on. > > > -- > > ________________________________ > toddgrimason*todd[ at ]slack.net > > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >
On 7/5/05, Todd Grimason <todd-cwT7Wi5Y1r1eoWH0uzbU5w@public.gmane.org> wrote:> * Corey Lawson [2005-07-05 14:36]:<applied chainsaw...>> I could totally be wrong on this, but I believe since rails runs in a > process model there''s no need for connection pooling. As each process > starts up it opens a DB connection - and then each process only handles > one request at a time, no threading. So there''s never any contention or > need for additional db handles.There still could be a need for database connection pooling, though, outside of this mechanism... What about adding stuff into AbstractConnector.rb and the other connectors so that it might be possible to package the connection layer into a "service" that runs on the db server, from which the webserver hosting the Rails app can then talk to, using Drb (or some other CORBA/RPC like mechanism)? (Yes, I know. code something up first).> And most opinions seem to be that using FastCGI in static mode is the > way to go, that is not having it/apache manage the number of FCGI procs > dynamically, but rather starting N procs upon startup. So your slow > Oracle connections would all be opened upon app startup (like a db conn > pool) and you shouldn''t have any connection time issues from then on....but can you dynamically increase/decrease the processes, then? There can be technical reasons to not want 1000 connections open on a database server (when a pool of 50-100 could manage things) at any given time... Depending on the db, the database connectors *could* (like SQL Server) allow the connection to time-out after a period of non-activity, assuming the processes not being used just sit idle. A J2EE app I used to have to manage did just this. It served several hundred simultaneous read/write app users via a J2EE connection pool of 5-20 Oracle connections (that was the WebLogic configuration for the db connection pool as I last remember it). I guess I''m thinking maybe too far down the road, but it will be something that eventually all the "enterprise" pundits will throw at Rails, along with "N-Tier", distributed databases (not a problem I suppose if Oracle 9i-RAC/10g is doing it for you), etc.> ________________________________ > toddgrimason*todd[ at ]slack.net > > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >
> I still would like to hear how many fcgi procs a site like backpack > uses, or any other high-traffic rails sites for that matter.I''ve found 10-15 FCGIs per machine to be a good number for Backpack/Basecamp. Some sites can do even less. Some need a bit more. You certainly don''t need hundreds, though. So each application server will need to maintain about 10-15 connections to the database. That seems like a reasonable number. Also, once the FCGI connection has established a connection it holds on to it. So there''s no overhead on recreating or managing a pool. I''d be interested in hearing real-life scenarios where 100s of FCGI processes per machine is desirable. -- David Heinemeier Hansson http://www.loudthinking.com -- Broadcasting Brain http://www.basecamphq.com -- Online project management http://www.backpackit.com -- Personal information manager http://www.rubyonrails.com -- Web-application framework