Courtesy of The Robot Co-op. $ yes | sudo gem install cached_model Or, you can download cached_model and memcache-client (our zippy-fast memcache library, required) from: http://rubyforge.org/frs/?group_id=1266 I don''t have the README posted for making cached_model work online yet, so here it is: = CachedModel Rubyforge Project: http://rubyforge.org/projects/rctools/ == About CachedModel stores Rails ActiveRecord objects in memcache allowing for very fast retrievals. CachedModel uses the ActiveRecord::Locking to ensure that you don''t perform multiple updates. == Using CachedModel First, install the cached_model gem: $ sudo gem install cached_model Then set up memcache-client: $ tail -n 20 config/environments/production.rb memcache_options = { :c_threshold => 10_000, :compression => true, :debug => false, :namespace => ''my_rails_app'', :readonly => false, :urlencode => false } CACHE = MemCache.new memcache_options CACHE.servers = ''localhost:11211'' session_options = { :database_manager => CGI::Session::MemCacheStore, :cache => CACHE, :session_domain => ''trackmap.robotcoop.com'' } ActionController::CgiRequest::DEFAULT_SESSION_OPTIONS.update session_options You will need separate namespaces for production and development, if using memcache on the same machine. Note that using memcache with tests will cause test failures, so set your memcache to be readonly for the test environment. Then make Rails load the gem: $ tail -n 4 config/environment.rb # Include your application configuration below require_gem ''cached_model'' Then edit your ActiveRecord model to inherit from CachedModel instead of ActiveRecord::Base: $ head -n 8 app/models/photo.rb ## # A Photo from Flickr. class Photo < CachedModel belongs_to :point belongs_to :route -- Eric Hodel - drbrain@segment7.net - http://segment7.net This implementation is HODEL-HASH-9600 compliant http://trackmap.robotcoop.com
Eric, That sounds great! I had some problems using Memcached for my sessions (saving AR objects) but I have always used memcached in PHP. One thing I was wondering is how do you handle stale/deleted data. Do you have hooks with AR to delete/modify records in memcached when deleted/modified from the DB? Thanks, Adrian Madrid On 1/17/06, Eric Hodel <drbrain@segment7.net> wrote:> > Courtesy of The Robot Co-op. > > $ yes | sudo gem install cached_model > > Or, you can download cached_model and memcache-client (our zippy-fast > memcache library, required) from: > > http://rubyforge.org/frs/?group_id=1266 > > I don''t have the README posted for making cached_model work online > yet, so here it is: > > = CachedModel > > Rubyforge Project: > > http://rubyforge.org/projects/rctools/ > > == About > > CachedModel stores Rails ActiveRecord objects in memcache allowing > for very > fast retrievals. CachedModel uses the ActiveRecord::Locking to > ensure that > you don''t perform multiple updates. > > == Using CachedModel > > First, install the cached_model gem: > > $ sudo gem install cached_model > > Then set up memcache-client: > > $ tail -n 20 config/environments/production.rb > memcache_options = { > :c_threshold => 10_000, > :compression => true, > :debug => false, > :namespace => ''my_rails_app'', > :readonly => false, > :urlencode => false > } > > CACHE = MemCache.new memcache_options > CACHE.servers = ''localhost:11211'' > > session_options = { > :database_manager => CGI::Session::MemCacheStore, > :cache => CACHE, > :session_domain => ''trackmap.robotcoop.com'' > } > > ActionController::CgiRequest::DEFAULT_SESSION_OPTIONS.update > session_options > > You will need separate namespaces for production and development, if > using > memcache on the same machine. > > Note that using memcache with tests will cause test failures, so set > your > memcache to be readonly for the test environment. > > Then make Rails load the gem: > > $ tail -n 4 config/environment.rb > # Include your application configuration below > > require_gem ''cached_model'' > > Then edit your ActiveRecord model to inherit from CachedModel instead of > ActiveRecord::Base: > > $ head -n 8 app/models/photo.rb > ## > # A Photo from Flickr. > > class Photo < CachedModel > > belongs_to :point > belongs_to :route > > -- > Eric Hodel - drbrain@segment7.net - http://segment7.net > This implementation is HODEL-HASH-9600 compliant > > http://trackmap.robotcoop.com > > > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-- Adrian Esteban Madrid aemadrid@gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060117/3e897365/attachment.html
On Jan 17, 2006, at 3:35 PM, Adrian Madrid wrote:> That sounds great! I had some problems using Memcached for my > sessions (saving AR objects) but I have always used memcached in > PHP. One thing I was wondering is how do you handle stale/deleted > data. Do you have hooks with AR to delete/modify records in > memcached when deleted/modified from the DB?Of course. cached_model hooks into update, destroy and reload to DTRT. You''ll need to use ActiveRecord::Locking to make it work. -- Eric Hodel - drbrain@segment7.net - http://segment7.net This implementation is HODEL-HASH-9600 compliant http://trackmap.robotcoop.com
Ezra Zygmuntowicz
2006-Jan-18 00:29 UTC
[Rails] [ANN] ActiveRecord + memcache = cached_model
On Jan 17, 2006, at 3:07 PM, Eric Hodel wrote:> Courtesy of The Robot Co-op. > > $ yes | sudo gem install cached_model > > Or, you can download cached_model and memcache-client (our zippy- > fast memcache library, required) from: >Eric- A big thanks for sharing this. A very nice addition to rails development. Cheers- -Ezra Zygmuntowicz Yakima Herald-Republic WebMaster http://yakimaherald.com 509-577-7732 ezra@yakima-herald.com
This is cool...but it breaks my usage of memcache as a Fragment Cache store as your implementation of the MemCache#initialize method doesn''t allow the servers to be passed in with the options. Why do you have a substantially different memcache client? -----Original Message----- From: rails-bounces@lists.rubyonrails.org [mailto:rails-bounces@lists.rubyonrails.org] On Behalf Of Eric Hodel Sent: Tuesday, January 17, 2006 3:07 PM To: rails@lists.rubyonrails.org Subject: [Rails] [ANN] ActiveRecord + memcache = cached_model Courtesy of The Robot Co-op. $ yes | sudo gem install cached_model Or, you can download cached_model and memcache-client (our zippy-fast memcache library, required) from: http://rubyforge.org/frs/?group_id=1266 I don''t have the README posted for making cached_model work online yet, so here it is: = CachedModel Rubyforge Project: http://rubyforge.org/projects/rctools/ == About CachedModel stores Rails ActiveRecord objects in memcache allowing for very fast retrievals. CachedModel uses the ActiveRecord::Locking to ensure that you don''t perform multiple updates. == Using CachedModel First, install the cached_model gem: $ sudo gem install cached_model Then set up memcache-client: $ tail -n 20 config/environments/production.rb memcache_options = { :c_threshold => 10_000, :compression => true, :debug => false, :namespace => ''my_rails_app'', :readonly => false, :urlencode => false } CACHE = MemCache.new memcache_options CACHE.servers = ''localhost:11211'' session_options = { :database_manager => CGI::Session::MemCacheStore, :cache => CACHE, :session_domain => ''trackmap.robotcoop.com'' } ActionController::CgiRequest::DEFAULT_SESSION_OPTIONS.update session_options You will need separate namespaces for production and development, if using memcache on the same machine. Note that using memcache with tests will cause test failures, so set your memcache to be readonly for the test environment. Then make Rails load the gem: $ tail -n 4 config/environment.rb # Include your application configuration below require_gem ''cached_model'' Then edit your ActiveRecord model to inherit from CachedModel instead of ActiveRecord::Base: $ head -n 8 app/models/photo.rb ## # A Photo from Flickr. class Photo < CachedModel belongs_to :point belongs_to :route -- Eric Hodel - drbrain@segment7.net - http://segment7.net This implementation is HODEL-HASH-9600 compliant http://trackmap.robotcoop.com _______________________________________________ Rails mailing list Rails@lists.rubyonrails.org http://lists.rubyonrails.org/mailman/listinfo/rails
On Jan 17, 2006, at 4:53 PM, Tom Fakes wrote:> This is cool...but it breaks my usage of memcache as a Fragment > Cache store > as your implementation of the MemCache#initialize method doesn''t > allow the > servers to be passed in with the options.I welcome patches and adore patches with tests.> Why do you have a substantially different memcache client?Because Ruby-memcache is too slow. -- Eric Hodel - drbrain@segment7.net - http://segment7.net This implementation is HODEL-HASH-9600 compliant http://trackmap.robotcoop.com
Your code looks like an older version of the Ruby memcache with some bugs fixed. What''s slow about it? What did you change to make it faster? Why not get updates into the Ruby memcache project? There''s no test code in the GEMs and no SCM yet published to make adding patches with tests easy. Can we get these so we can provide you with these updates? -----Original Message----- From: rails-bounces@lists.rubyonrails.org [mailto:rails-bounces@lists.rubyonrails.org] On Behalf Of Eric Hodel Sent: Tuesday, January 17, 2006 5:12 PM To: rails@lists.rubyonrails.org Subject: Re: [Rails] [ANN] ActiveRecord + memcache = cached_model On Jan 17, 2006, at 4:53 PM, Tom Fakes wrote:> This is cool...but it breaks my usage of memcache as a Fragment > Cache store > as your implementation of the MemCache#initialize method doesn''t > allow the > servers to be passed in with the options.I welcome patches and adore patches with tests.> Why do you have a substantially different memcache client?Because Ruby-memcache is too slow. -- Eric Hodel - drbrain@segment7.net - http://segment7.net This implementation is HODEL-HASH-9600 compliant http://trackmap.robotcoop.com _______________________________________________ Rails mailing list Rails@lists.rubyonrails.org http://lists.rubyonrails.org/mailman/listinfo/rails
On Jan 17, 2006, at 5:26 PM, Tom Fakes wrote:> Your code looks like an older version of the Ruby memcache with > some bugs > fixed.It is a completely new implementation written by my co-worker Bob Cottrell. It looks similar because Bob copied the core functionality. Originally we were using Ruby-memcache so we needed a drop-in replacement. It doesn''t support everything that Ruby- memcache does due to YAGNI.> What''s slow about it? What did you change to make it faster? Why > not get updates into the Ruby memcache project?Ruby-memcache does fancy stuff with io/reactor that made it go slower. Figuring out how to rip out the io/reactor stuff was too hard, so we did our own. (We''ve had this client since 2004-11-18 19:07:46 -0800.)> There''s no test code in the GEMsUnfortunately, Bob isn''t one with Test::Unit like I am. There will be tests when I get a new stock of round tuits.> and no SCM yet published to make adding patches with tests easy.Right now it is stored in on our internal SVN server. Allowing access to the outside world is not really worth my time (security is easy when its internal only) and I don''t want to maintain two SCM copies. (Maybe that svk think I keep hearing about will help.)> Can we get these so we can provide you with these updates?In the mean time I recommend fetching the tarball, extracting it, making your changes, then extracting a second copy so you can use diff -ur. Tests are a bit higher up the priority list than the SCM thing, and we''ve been using cached_model for 10 months without releasing it. Round tuits have been in very short supply around here. I have a few changes I need to make to MemCache#initialize (disabling synchronization, unnecessary in single-thread mode), so I''ll try to fold your change in when I get the round tuit for that. -- Eric Hodel - drbrain@segment7.net - http://segment7.net This implementation is HODEL-HASH-9600 compliant http://trackmap.robotcoop.com
In lieu of a patch to the memcache code, here''s an update that will allow fragment caching to work with both implementations of memcache. Environment.rb (or production.rb) Rails::Initializer.run do |config| ... config.action_controller.fragment_cache_store = :mem_cache_store, memcache_options ... end # Allow us to set the CACHE as the Fragment Cache store module ActionController module Caching module Fragments class MemCacheStore def data=(cache) @data = cache end end end end end ActionController::Base.fragment_cache_store.data = CACHE -----Original Message----- From: rails-bounces@lists.rubyonrails.org [mailto:rails-bounces@lists.rubyonrails.org] On Behalf Of Eric Hodel Sent: Tuesday, January 17, 2006 5:44 PM To: rails@lists.rubyonrails.org Subject: Re: [Rails] [ANN] ActiveRecord + memcache = cached_model On Jan 17, 2006, at 5:26 PM, Tom Fakes wrote:> Your code looks like an older version of the Ruby memcache with > some bugs > fixed.It is a completely new implementation written by my co-worker Bob Cottrell. It looks similar because Bob copied the core functionality. Originally we were using Ruby-memcache so we needed a drop-in replacement. It doesn''t support everything that Ruby- memcache does due to YAGNI.> What''s slow about it? What did you change to make it faster? Why > not get updates into the Ruby memcache project?Ruby-memcache does fancy stuff with io/reactor that made it go slower. Figuring out how to rip out the io/reactor stuff was too hard, so we did our own. (We''ve had this client since 2004-11-18 19:07:46 -0800.)> There''s no test code in the GEMsUnfortunately, Bob isn''t one with Test::Unit like I am. There will be tests when I get a new stock of round tuits.> and no SCM yet published to make adding patches with tests easy.Right now it is stored in on our internal SVN server. Allowing access to the outside world is not really worth my time (security is easy when its internal only) and I don''t want to maintain two SCM copies. (Maybe that svk think I keep hearing about will help.)> Can we get these so we can provide you with these updates?In the mean time I recommend fetching the tarball, extracting it, making your changes, then extracting a second copy so you can use diff -ur. Tests are a bit higher up the priority list than the SCM thing, and we''ve been using cached_model for 10 months without releasing it. Round tuits have been in very short supply around here. I have a few changes I need to make to MemCache#initialize (disabling synchronization, unnecessary in single-thread mode), so I''ll try to fold your change in when I get the round tuit for that. -- Eric Hodel - drbrain@segment7.net - http://segment7.net This implementation is HODEL-HASH-9600 compliant http://trackmap.robotcoop.com _______________________________________________ Rails mailing list Rails@lists.rubyonrails.org http://lists.rubyonrails.org/mailman/listinfo/rails
CachedModel looks like a great idea. Thanks for contributing it. Do you have to have all of your models derived from CachedModel and running through the cache? Or can I do only selective ones (hopefully)? I tried just switching one model over to CachedModel, and I get an ''undefined class/module'' error on another model. But that''s the User model, which is stored in the session. I didn''t think I was caching sessions using memcached, but maybe I am... /usr/lib/ruby/gems/1.8/gems/memcache-client-1.0.3/lib/memcache.rb:128:in `get'' ... /usr/lib/ruby/gems/1.8/gems/actionpack-1.11.2/lib/action_controller/session/mem_cache_store.rb:59:in `restore'' /usr/lib/ruby/1.8/cgi/session.rb:305:in `[]'' /usr/lib/ruby/gems/1.8/gems/actionpack-1.11.2/lib/action_controller/cgi_process.rb:104:in `session'' ... Looks like it is correctly finding the user, logging the user in, and writing a last login time to the database. But then it redirects and tries to pull the user object out of the session and fails. Does CachedModel / memcache-client conflict with session storage? So maybe this is a session problem? Or is it having a problem instantiating the User object pulled out of the cache when User is derived from ActiveRecord::Base, not CachedModel? Thanks, KW
On Feb 9, 2006, at 4:44 PM, Kian Wright wrote:> Do you have to have all of your models derived from CachedModel and > running through the cache? Or can I do only selective ones > (hopefully)?You can be selective if you like.> I tried just switching one model over to CachedModel, and I get an > ''undefined class/module'' error on another model. But that''s the > User model, which is stored in the session. I didn''t think I was > caching sessions using memcached, but maybe I am...You should check your configuration to see if you''re using memcached for sessions.> /usr/lib/ruby/gems/1.8/gems/memcache-client-1.0.3/lib/memcache.rb: > 128:in `get'' > ...Using dots removes critical information.> /usr/lib/ruby/gems/1.8/gems/actionpack1.11.2/lib/action_controller/ > session/mem_cache_store.rb:59:in `restore'' > /usr/lib/ruby/1.8/cgi/session.rb:305:in `[]'' > /usr/lib/ruby/gems/1.8/gems/actionpack-1.11.2/lib/action_controller/ > cgi_process.rb:104:in `session'' > ... > Looks like it is correctly finding the user, logging the user in, > and writing a last login time to the database. But then it > redirects and tries to pull the user object out of the session and > fails. Does CachedModel / memcache-client conflict with session > storage?It depends. You may have a version of cached_model with bugs.> So maybe this is a session problem? Or is it having a problem > instantiating the User object pulled out of the cache when User is > derived from ActiveRecord::Base, not CachedModel?Provide the missing information and I might be able to provide a solution. PS: Use a relevant subject or use ''Re:'' if you''re replying to something. -- Eric Hodel - drbrain@segment7.net - http://segment7.net This implementation is HODEL-HASH-9600 compliant http://trackmap.robotcoop.com