Hello,
Due to the Trac situation, I am forwarding this patch along to this list in
the hopes of elicting some feedback. I will be sure to open a proper ticket
when possible.
I have done some extensive work on active record to allow for databases to
be defined in terms of "connection pools" rather than simply one
database.
Most of the work was completed in connection_specification.rb and is done at
this level to be compatible with any backend you choose. Also this patch
maintains full compatibiltity with the current paradigm so no changes are
necessary to current applications/documentation.
Defining a database pool is very easy and follows a common sense
convention. I will assume you are running with RAILS_ENV=development for
this example. You first define your connections as you always have in
database.yml . Your are then able to able to define
development_read_pool: db1, db2, db3
development_write_pool: db3, db4, db5 (In general, the name is
RAILS_ENV_write_pool so you can test your clusters in development and
production with no config changes)
where each is a set of connection names that you have already defined.
ActiveRecord::Base.connection will then return connections from the
appropriate pool using round-robin when more than one connection is
available to it. This is handy if you hvae a high traffic website for
example and you want to load balance over several slave servers for reading
while writing to the one master server consistently. The syntax for
.connection is as follow
ExampleModel.connection # default "compatibility behavior"
always
returns a write connection.
ExampleModel.connection(:read) # Return a connection from the read pool
ExampleModel.connection (:write) # Return a connection from the write pool
I have also changed a few of the functions in base.rb to utilize the
correction pool (for example, find_by_sql calls connection(:read)) This
again makes the patch seamless to end applications while allowing them to
use the new functionality.
A patch aganist the lastest CVS is attached. Some rough notes on
implementation are also include below - there not 100% complete but give a
good idea of what was changed.
This currently passes an ActiveRecord rake with flying colors.
Implementation:
1. Added two arrays to act as pools for connections related to this
connection.
@@read_connection_pool = {}
@@write_connection_pool = {}
Each is later defined as an array such that
@@read_connection_pool[name] represenets all of the read connections
available to the current
class. This allows us to stay fully backward compatiable with the
old methodology where name
is @active_connection_name
2. Define 2 variables to track index of last used connection. This is used
when doing round-robin
@@last_read_connection = 0
@@last_write_connection = 0
3. Define a function for appending connections to pools
(append_spec_to_connection_pools spec).
If the config has an attribute read_only == true then it is only entered
into the read_connection_pool
and vice versa for the write_connection_pool. @@defined_connections
also is defined at the same time.
@@write_connection_pool[spec.object_id] always equals
@@defined_connections[spec.object_id]
5. Define a function establish_connection_pools which looks for two
variables to be
set in the configuration:
RAILS_ENV_read_connection_pool
RAILS_ENV_write_conenction_pool
Normally in Rails, this would setup in the YAML file. Each variable is
a comma delimited
list of connections to use, one for read the other for write.
6. Define a function clear_connection_pool which clears the connection
pools.
7. Modify establish_connection
a. When passed nil, call establish_connection_pools(RAILS_ENV)
b. When passed a ConnectionSpecification, clear all connection pools as
well as the
active_connection_name
8. Move code from establish connection to ConnectionSpecification
constructor so an object can be made
out of any spec thats passed to it. (Avoiding breakage of DRY
principle)
9. Remove settings @@defined_connections[name] in establish_connection.
This is now done by
calling
10. Define two functions, round robin read and round robin write which
returns AbstractAdapters from the various pools.
11. Modify retreive connection so that it takes an id of an object that
should be in @@defined_connections
Remove it''s dependency on connection= as this will break things.
12. Modify connection= to call establish_connection on
ConnectionSpecification.
13. Moidfy self.remove_connection to call clear_connection_pool
14. Modify active_connection_name to check if a pool exists instead of
defined_connections
Patch attached is aganist whats in the subversion repo.
Best Regards,
Stephen Blackstone
_______________________________________________
Rails-core mailing list
Rails-core@lists.rubyonrails.org
http://lists.rubyonrails.org/mailman/listinfo/rails-core
Hello,
Due to the Trac situation, I am forwarding this patch along to this list in
the hopes of elicting some feedback. I will be sure to open a proper ticket
when possible.
I have done some extensive work on active record to allow for databases to
be defined in terms of "connection pools" rather than simply one
database.
Most of the work was completed in connection_specification.rb and is done at
this level to be compatible with any backend you choose. Also this patch
maintains full compatibiltity with the current paradigm so no changes are
necessary to current applications/documentation.
Defining a database pool is very easy and follows a common sense
convention. I will assume you are running with RAILS_ENV=development for
this example. You first define your connections as you always have in
database.yml . Your are then able to able to define
development_read_pool: db1, db2, db3
development_write_pool: db3, db4, db5 (In general, the name is
RAILS_ENV_write_pool so you can test your clusters in development and
production with no config changes)
where each is a set of connection names that you have already defined.
ActiveRecord::Base.connection will then return connections from the
appropriate pool using round-robin when more than one connection is
available to it. This is handy if you hvae a high traffic website for
example and you want to load balance over several slave servers for reading
while writing to the one master server consistently. The syntax for
.connection is as follow
ExampleModel.connection # default "compatibility behavior"
always
returns a write connection.
ExampleModel.connection(:read) # Return a connection from the read pool
ExampleModel.connection (:write) # Return a connection from the write pool
I have also changed a few of the functions in base.rb to utilize the
correction pool (for example, find_by_sql calls connection(:read)) This
again makes the patch seamless to end applications while allowing them to
use the new functionality.
A patch aganist the lastest CVS is attached. Some rough notes on
implementation are also include below - there not 100% complete but give a
good idea of what was changed.
This currently passes an ActiveRecord rake with flying colors.
Implementation:
1. Added two arrays to act as pools for connections related to this
connection.
@@read_connection_pool = {}
@@write_connection_pool = {}
Each is later defined as an array such that
@@read_connection_pool[name] represenets all of the read connections
available to the current
class. This allows us to stay fully backward compatiable with the
old methodology where name
is @active_connection_name
2. Define 2 variables to track index of last used connection. This is used
when doing round-robin
@@last_read_connection = 0
@@last_write_connection = 0
3. Define a function for appending connections to pools
(append_spec_to_connection_pools spec).
If the config has an attribute read_only == true then it is only entered
into the read_connection_pool
and vice versa for the write_connection_pool. @@defined_connections
also is defined at the same time.
@@write_connection_pool[spec.object_id] always equals
@@defined_connections[spec.object_id]
5. Define a function establish_connection_pools which looks for two
variables to be
set in the configuration:
RAILS_ENV_read_connection_pool
RAILS_ENV_write_conenction_pool
Normally in Rails, this would setup in the YAML file. Each variable is
a comma delimited
list of connections to use, one for read the other for write.
6. Define a function clear_connection_pool which clears the connection
pools.
7. Modify establish_connection
a. When passed nil, call establish_connection_pools(RAILS_ENV)
b. When passed a ConnectionSpecification, clear all connection pools as
well as the
active_connection_name
8. Move code from establish connection to ConnectionSpecification
constructor so an object can be made
out of any spec thats passed to it. (Avoiding breakage of DRY
principle)
9. Remove settings @@defined_connections[name] in establish_connection.
This is now done by
calling
10. Define two functions, round robin read and round robin write which
returns AbstractAdapters from the various pools.
11. Modify retreive connection so that it takes an id of an object that
should be in @@defined_connections
Remove it''s dependency on connection= as this will break things.
12. Modify connection= to call establish_connection on
ConnectionSpecification.
13. Moidfy self.remove_connection to call clear_connection_pool
14. Modify active_connection_name to check if a pool exists instead of
defined_connections
Patch attached is aganist whats in the subversion repo.
Best Regards,
Stephen Blackstone
_______________________________________________
Rails-core mailing list
Rails-core@lists.rubyonrails.org
http://lists.rubyonrails.org/mailman/listinfo/rails-core