Backgroundrb devel - Aug 2008 - multiple database design question

Hi all. I have a few questions, but I''ll start a thread per question.  
My first is a design issue that''s fairly specific to my application,  
but I thought someone might be willing to give out some insights.

My application has multiple databases, one per client, and I hijack  
the database connection at the beginning of a request and connect it  
to the appropriate database. So, I have to pass the database to any  
workers so that they know which database to operate on. I have two  
choices here:

1) Have a single worker, pass in the database to all worker methods,  
and have the method connect to the correct database before beginning  
the real processing

2) Create multiple workers, one per database, and have them connect to  
the appropriate database when initialized via the ''create''
method.
Rails can then call methods on the appropriate database specific  
worker. without having to pass the database in to each method,  
although it will have to use the database as a worker key when getting  
the correct worker.

I like the organization of the second method, but the simplicity of  
the first. I''m thinking I''ll go with the first. Any thoughts?

On Fri, Aug 15, 2008 at 12:32 AM, Woody Peterson
<woody at crystalcommerce.com> wrote:
> Hi all. I have a few questions, but I''ll start a thread per
question. My
> first is a design issue that''s fairly specific to my application,
but I
> thought someone might be willing to give out some insights.
>
> My application has multiple databases, one per client, and I hijack the
> database connection at the beginning of a request and connect it to the
> appropriate database. So, I have to pass the database to any workers so
that
> they know which database to operate on. I have two choices here:
>
> 1) Have a single worker, pass in the database to all worker methods, and
> have the method connect to the correct database before beginning the real
> processing
>
> 2) Create multiple workers, one per database, and have them connect to the
> appropriate database when initialized via the ''create''
method. Rails can
> then call methods on the appropriate database specific worker. without
> having to pass the database in to each method, although it will have to use
> the database as a worker key when getting the correct worker.
>
> I like the organization of the second method, but the simplicity of the
> first. I''m thinking I''ll go with the first. Any thoughts?
Second version, certainly looks cleaner, besides, I wouldn''t know, how
will you be passing db connections to worker.

On Aug 14, 2008, at 8:13 PM, hemant wrote:
> On Fri, Aug 15, 2008 at 12:32 AM, Woody Peterson
> <woody at crystalcommerce.com> wrote:
>> Hi all. I have a few questions, but I''ll start a thread per  
>> question. My
>> first is a design issue that''s fairly specific to my
application,
>> but I
>> thought someone might be willing to give out some insights.
>>
>> My application has multiple databases, one per client, and I hijack  
>> the
>> database connection at the beginning of a request and connect it to  
>> the
>> appropriate database. So, I have to pass the database to any  
>> workers so that
>> they know which database to operate on. I have two choices here:
>>
>> 1) Have a single worker, pass in the database to all worker  
>> methods, and
>> have the method connect to the correct database before beginning  
>> the real
>> processing
>>
>> 2) Create multiple workers, one per database, and have them connect  
>> to the
>> appropriate database when initialized via the
''create'' method.
>> Rails can
>> then call methods on the appropriate database specific worker.  
>> without
>> having to pass the database in to each method, although it will  
>> have to use
>> the database as a worker key when getting the correct worker.
>>
>> I like the organization of the second method, but the simplicity of  
>> the
>> first. I''m thinking I''ll go with the first. Any
thoughts?
>
> Second version, certainly looks cleaner, besides, I wouldn''t know,
how
> will you be passing db connections to worker.
I wouldn''t pass the connection, just the database name via :arg =>  
{:database => ActiveRecord::Base.connection.current_database}, then  
call our hijack method to create a new connection once inside the  
worker method. The second method is cleaner, but on a resource level,  
what happens when you have lots of workers? If successful, we''ll  
eventually have hundreds of clients, thus hundreds of workers. Seems  
to reason that it''d take more resources for that scenario. A benefit  
though is that for persistent jobs each worker would poll it''s  
respective database, and I wouldn''t have to implement some special  
poll-all-databases code for the one worker.

Anyways, thanks for all your feedback!

-Woody