thr3ads.net - Rails - [Rails] [ANN] ActiveRecord + memcache = cached

If this information is useful, please help other people find it:
Share via:

Eric Hodel

2006-Jan-17 23:07 UTC

[Rails] [ANN] ActiveRecord + memcache = cached_model

Courtesy of The Robot Co-op.

$ yes | sudo gem install cached_model

Or, you can download cached_model and memcache-client (our zippy-fast  
memcache library, required) from:

http://rubyforge.org/frs/?group_id=1266

I don''t have the README posted for making cached_model work online  
yet, so here it is:

= CachedModel

Rubyforge Project:

http://rubyforge.org/projects/rctools/

== About

CachedModel stores Rails ActiveRecord objects in memcache allowing  
for very
fast retrievals.  CachedModel uses the ActiveRecord::Locking to  
ensure that
you don''t perform multiple updates.

== Using CachedModel

First, install the cached_model gem:

   $ sudo gem install cached_model

Then set up memcache-client:

   $ tail -n 20  config/environments/production.rb
   memcache_options = {
     :c_threshold => 10_000,
     :compression => true,
     :debug => false,
     :namespace => ''my_rails_app'',
     :readonly => false,
     :urlencode => false
   }

   CACHE = MemCache.new memcache_options
   CACHE.servers = ''localhost:11211''

   session_options = {
     :database_manager => CGI::Session::MemCacheStore,
     :cache => CACHE,
     :session_domain => ''trackmap.robotcoop.com''
   }

   ActionController::CgiRequest::DEFAULT_SESSION_OPTIONS.update  
session_options

You will need separate namespaces for production and development, if  
using
memcache on the same machine.

Note that using memcache with tests will cause test failures, so set  
your
memcache to be readonly for the test environment.

Then make Rails load the gem:

   $ tail -n 4 config/environment.rb
   # Include your application configuration below

   require_gem ''cached_model''

Then edit your ActiveRecord model to inherit from CachedModel instead of
ActiveRecord::Base:

   $ head -n 8 app/models/photo.rb
   ##
   # A Photo from Flickr.

   class Photo < CachedModel

     belongs_to :point
     belongs_to :route

-- 
Eric Hodel - drbrain@segment7.net - http://segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com

Adrian Madrid

2006-Jan-17 23:35 UTC

head link

[Rails] [ANN] ActiveRecord + memcache = cached_model

Eric,

That sounds great! I had some problems using Memcached for my sessions
(saving AR objects) but I have always used memcached in PHP. One thing I was
wondering is how do you handle stale/deleted data. Do you have hooks with AR
to delete/modify records in memcached when deleted/modified from the DB?

Thanks,


Adrian Madrid


On 1/17/06, Eric Hodel <drbrain@segment7.net>
wrote:>
> Courtesy of The Robot Co-op.
>
> $ yes | sudo gem install cached_model
>
> Or, you can download cached_model and memcache-client (our zippy-fast
> memcache library, required) from:
>
> http://rubyforge.org/frs/?group_id=1266
>
> I don''t have the README posted for making cached_model work online
> yet, so here it is:
>
> = CachedModel
>
> Rubyforge Project:
>
> http://rubyforge.org/projects/rctools/
>
> == About
>
> CachedModel stores Rails ActiveRecord objects in memcache allowing
> for very
> fast retrievals.  CachedModel uses the ActiveRecord::Locking to
> ensure that
> you don''t perform multiple updates.
>
> == Using CachedModel
>
> First, install the cached_model gem:
>
>    $ sudo gem install cached_model
>
> Then set up memcache-client:
>
>    $ tail -n 20  config/environments/production.rb
>    memcache_options = {
>      :c_threshold => 10_000,
>      :compression => true,
>      :debug => false,
>      :namespace => ''my_rails_app'',
>      :readonly => false,
>      :urlencode => false
>    }
>
>    CACHE = MemCache.new memcache_options
>    CACHE.servers = ''localhost:11211''
>
>    session_options = {
>      :database_manager => CGI::Session::MemCacheStore,
>      :cache => CACHE,
>      :session_domain => ''trackmap.robotcoop.com''
>    }
>
>    ActionController::CgiRequest::DEFAULT_SESSION_OPTIONS.update
> session_options
>
> You will need separate namespaces for production and development, if
> using
> memcache on the same machine.
>
> Note that using memcache with tests will cause test failures, so set
> your
> memcache to be readonly for the test environment.
>
> Then make Rails load the gem:
>
>    $ tail -n 4 config/environment.rb
>    # Include your application configuration below
>
>    require_gem ''cached_model''
>
> Then edit your ActiveRecord model to inherit from CachedModel instead of
> ActiveRecord::Base:
>
>    $ head -n 8 app/models/photo.rb
>    ##
>    # A Photo from Flickr.
>
>    class Photo < CachedModel
>
>      belongs_to :point
>      belongs_to :route
>
> --
> Eric Hodel - drbrain@segment7.net - http://segment7.net
> This implementation is HODEL-HASH-9600 compliant
>
> http://trackmap.robotcoop.com
>
>
> _______________________________________________
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>


--
Adrian Esteban Madrid
aemadrid@gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://wrath.rubyonrails.org/pipermail/rails/attachments/20060117/3e897365/attachment.html

Eric Hodel

2006-Jan-17 23:48 UTC

head link

[Rails] [ANN] ActiveRecord + memcache = cached_model

On Jan 17, 2006, at 3:35 PM, Adrian Madrid wrote:
> That sounds great! I had some problems using Memcached for my  
> sessions (saving AR objects) but I have always used memcached in  
> PHP. One thing I was wondering is how do you handle stale/deleted  
> data. Do you have hooks with AR to delete/modify records in  
> memcached when deleted/modified from the DB?
Of course.  cached_model hooks into update, destroy and reload to  
DTRT.  You''ll need to use ActiveRecord::Locking to make it work.

-- 
Eric Hodel - drbrain@segment7.net - http://segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com

Ezra Zygmuntowicz

2006-Jan-18 00:29 UTC

head link

[Rails] [ANN] ActiveRecord + memcache = cached_model

On Jan 17, 2006, at 3:07 PM, Eric Hodel wrote:
> Courtesy of The Robot Co-op.
>
> $ yes | sudo gem install cached_model
>
> Or, you can download cached_model and memcache-client (our zippy- 
> fast memcache library, required) from:
>

Eric-

	A big thanks for sharing this. A very nice addition to rails  
development.

Cheers-
-Ezra Zygmuntowicz
Yakima Herald-Republic
WebMaster
http://yakimaherald.com
509-577-7732
ezra@yakima-herald.com

Tom Fakes

2006-Jan-18 00:52 UTC

head link

[Rails] [ANN] ActiveRecord + memcache = cached_model

This is cool...but it breaks my usage of memcache as a Fragment Cache store
as your implementation of the MemCache#initialize method doesn''t allow
the
servers to be passed in with the options.

Why do you have a substantially different memcache client?

-----Original Message-----
From: rails-bounces@lists.rubyonrails.org
[mailto:rails-bounces@lists.rubyonrails.org] On Behalf Of Eric Hodel
Sent: Tuesday, January 17, 2006 3:07 PM
To: rails@lists.rubyonrails.org
Subject: [Rails] [ANN] ActiveRecord + memcache = cached_model

Courtesy of The Robot Co-op.

$ yes | sudo gem install cached_model

Or, you can download cached_model and memcache-client (our zippy-fast  
memcache library, required) from:

http://rubyforge.org/frs/?group_id=1266

I don''t have the README posted for making cached_model work online  
yet, so here it is:

= CachedModel

Rubyforge Project:

http://rubyforge.org/projects/rctools/

== About

CachedModel stores Rails ActiveRecord objects in memcache allowing  
for very
fast retrievals.  CachedModel uses the ActiveRecord::Locking to  
ensure that
you don''t perform multiple updates.

== Using CachedModel

First, install the cached_model gem:

   $ sudo gem install cached_model

Then set up memcache-client:

   $ tail -n 20  config/environments/production.rb
   memcache_options = {
     :c_threshold => 10_000,
     :compression => true,
     :debug => false,
     :namespace => ''my_rails_app'',
     :readonly => false,
     :urlencode => false
   }

   CACHE = MemCache.new memcache_options
   CACHE.servers = ''localhost:11211''

   session_options = {
     :database_manager => CGI::Session::MemCacheStore,
     :cache => CACHE,
     :session_domain => ''trackmap.robotcoop.com''
   }

   ActionController::CgiRequest::DEFAULT_SESSION_OPTIONS.update  
session_options

You will need separate namespaces for production and development, if  
using
memcache on the same machine.

Note that using memcache with tests will cause test failures, so set  
your
memcache to be readonly for the test environment.

Then make Rails load the gem:

   $ tail -n 4 config/environment.rb
   # Include your application configuration below

   require_gem ''cached_model''

Then edit your ActiveRecord model to inherit from CachedModel instead of
ActiveRecord::Base:

   $ head -n 8 app/models/photo.rb
   ##
   # A Photo from Flickr.

   class Photo < CachedModel

     belongs_to :point
     belongs_to :route

-- 
Eric Hodel - drbrain@segment7.net - http://segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com


_______________________________________________
Rails mailing list
Rails@lists.rubyonrails.org
http://lists.rubyonrails.org/mailman/listinfo/rails

Eric Hodel

2006-Jan-18 01:12 UTC

head link

[Rails] [ANN] ActiveRecord + memcache = cached_model

On Jan 17, 2006, at 4:53 PM, Tom Fakes wrote:
> This is cool...but it breaks my usage of memcache as a Fragment  
> Cache store
> as your implementation of the MemCache#initialize method doesn''t  
> allow the
> servers to be passed in with the options.
I welcome patches and adore patches with tests.
> Why do you have a substantially different memcache client?
Because Ruby-memcache is too slow.

-- 
Eric Hodel - drbrain@segment7.net - http://segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com

Tom Fakes

2006-Jan-18 01:25 UTC

head link

[Rails] [ANN] ActiveRecord + memcache = cached_model

Your code looks like an older version of the Ruby memcache with some bugs
fixed.  What''s slow about it?  What did you change to make it faster? 
Why
not get updates into the Ruby memcache project?


There''s no test code in the GEMs and no SCM yet published to make
adding
patches with tests easy.  Can we get these so we can provide you with these
updates?



-----Original Message-----
From: rails-bounces@lists.rubyonrails.org
[mailto:rails-bounces@lists.rubyonrails.org] On Behalf Of Eric Hodel
Sent: Tuesday, January 17, 2006 5:12 PM
To: rails@lists.rubyonrails.org
Subject: Re: [Rails] [ANN] ActiveRecord + memcache = cached_model

On Jan 17, 2006, at 4:53 PM, Tom Fakes wrote:
> This is cool...but it breaks my usage of memcache as a Fragment  
> Cache store
> as your implementation of the MemCache#initialize method doesn''t  
> allow the
> servers to be passed in with the options.
I welcome patches and adore patches with tests.
> Why do you have a substantially different memcache client?
Because Ruby-memcache is too slow.

-- 
Eric Hodel - drbrain@segment7.net - http://segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com


_______________________________________________
Rails mailing list
Rails@lists.rubyonrails.org
http://lists.rubyonrails.org/mailman/listinfo/rails

Eric Hodel

2006-Jan-18 01:44 UTC

head link

[Rails] [ANN] ActiveRecord + memcache = cached_model

On Jan 17, 2006, at 5:26 PM, Tom Fakes wrote:
> Your code looks like an older version of the Ruby memcache with  
> some bugs
> fixed.
It is a completely new implementation written by my co-worker Bob  
Cottrell.  It looks similar because Bob copied the core  
functionality.  Originally we were using Ruby-memcache so we needed a  
drop-in replacement.  It doesn''t support everything that Ruby- 
memcache does due to YAGNI.
> What''s slow about it?  What did you change to make it faster?  Why
> not get updates into the Ruby memcache project?
Ruby-memcache does fancy stuff with io/reactor that made it go  
slower.  Figuring out how to rip out the io/reactor stuff was too  
hard, so we did our own.  (We''ve had this client since 2004-11-18  
19:07:46 -0800.)
> There''s no test code in the GEMs
Unfortunately, Bob isn''t one with Test::Unit like I am.  There will  
be tests when I get a new stock of round tuits.
> and no SCM yet published to make adding patches with tests easy.
Right now it is stored in on our internal SVN server.  Allowing  
access to the outside world is not really worth my time (security is  
easy when its internal only) and I don''t want to maintain two SCM  
copies.  (Maybe that svk think I keep hearing about will help.)
> Can we get these so we can provide you with these updates?
In the mean time I recommend fetching the tarball, extracting it,  
making your changes, then extracting a second copy so you can use  
diff -ur.

Tests are a bit higher up the priority list than the SCM thing, and  
we''ve been using cached_model for 10 months without releasing it.   
Round tuits have been in very short supply around here.

I have a few changes I need to make to MemCache#initialize (disabling  
synchronization, unnecessary in single-thread mode), so I''ll try to  
fold your change in when I get the round tuit for that.

-- 
Eric Hodel - drbrain@segment7.net - http://segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com

Tom Fakes

2006-Jan-18 02:07 UTC

head link

[Rails] [ANN] ActiveRecord + memcache = cached_model

In lieu of a patch to the memcache code, here''s an update that will
allow
fragment caching to work with both implementations of memcache.

Environment.rb (or production.rb)

Rails::Initializer.run do |config|
  ...
  config.action_controller.fragment_cache_store = :mem_cache_store,
memcache_options
  ...
end

# Allow us to set the CACHE as the Fragment Cache store
module ActionController
  module Caching
    module Fragments
      class MemCacheStore
        def data=(cache)
          @data = cache
        end
      end
    end
  end
end

ActionController::Base.fragment_cache_store.data = CACHE


-----Original Message-----
From: rails-bounces@lists.rubyonrails.org
[mailto:rails-bounces@lists.rubyonrails.org] On Behalf Of Eric Hodel
Sent: Tuesday, January 17, 2006 5:44 PM
To: rails@lists.rubyonrails.org
Subject: Re: [Rails] [ANN] ActiveRecord + memcache = cached_model

On Jan 17, 2006, at 5:26 PM, Tom Fakes wrote:
> Your code looks like an older version of the Ruby memcache with  
> some bugs
> fixed.
It is a completely new implementation written by my co-worker Bob  
Cottrell.  It looks similar because Bob copied the core  
functionality.  Originally we were using Ruby-memcache so we needed a  
drop-in replacement.  It doesn''t support everything that Ruby- 
memcache does due to YAGNI.
> What''s slow about it?  What did you change to make it faster?  Why
> not get updates into the Ruby memcache project?
Ruby-memcache does fancy stuff with io/reactor that made it go  
slower.  Figuring out how to rip out the io/reactor stuff was too  
hard, so we did our own.  (We''ve had this client since 2004-11-18  
19:07:46 -0800.)
> There''s no test code in the GEMs
Unfortunately, Bob isn''t one with Test::Unit like I am.  There will  
be tests when I get a new stock of round tuits.
> and no SCM yet published to make adding patches with tests easy.
Right now it is stored in on our internal SVN server.  Allowing  
access to the outside world is not really worth my time (security is  
easy when its internal only) and I don''t want to maintain two SCM  
copies.  (Maybe that svk think I keep hearing about will help.)
> Can we get these so we can provide you with these updates?
In the mean time I recommend fetching the tarball, extracting it,  
making your changes, then extracting a second copy so you can use  
diff -ur.

Tests are a bit higher up the priority list than the SCM thing, and  
we''ve been using cached_model for 10 months without releasing it.   
Round tuits have been in very short supply around here.

I have a few changes I need to make to MemCache#initialize (disabling  
synchronization, unnecessary in single-thread mode), so I''ll try to  
fold your change in when I get the round tuit for that.

-- 
Eric Hodel - drbrain@segment7.net - http://segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com


_______________________________________________
Rails mailing list
Rails@lists.rubyonrails.org
http://lists.rubyonrails.org/mailman/listinfo/rails

Kian Wright

2006-Feb-10 00:44 UTC

head link

[Rails] [ANN] ActiveRecord + memcache = cached_model

CachedModel looks like a great idea. Thanks for contributing it.

Do you have to have all of your models derived from CachedModel and running
through the cache? Or
can I do only selective ones (hopefully)?

I tried just switching one model over to CachedModel, and I get an
''undefined class/module'' error
on another model. But that''s the User model, which is stored in the
session. I didn''t think I was
caching sessions using memcached, but maybe I am...

/usr/lib/ruby/gems/1.8/gems/memcache-client-1.0.3/lib/memcache.rb:128:in
`get''
...
/usr/lib/ruby/gems/1.8/gems/actionpack-1.11.2/lib/action_controller/session/mem_cache_store.rb:59:in
`restore''
/usr/lib/ruby/1.8/cgi/session.rb:305:in `[]''
/usr/lib/ruby/gems/1.8/gems/actionpack-1.11.2/lib/action_controller/cgi_process.rb:104:in
`session''
...
Looks like it is correctly finding the user, logging the user in, and writing a
last login time to
the database. But then it redirects and tries to pull the user object out of the
session and
fails. Does CachedModel / memcache-client conflict with session storage?

So maybe this is a session problem? Or is it having a problem instantiating the
User object pulled
out of the cache when User is derived from ActiveRecord::Base, not CachedModel?

Thanks,
KW

Eric Hodel

2006-Feb-12 07:18 UTC

head link

[Rails] [ANN] ActiveRecord + memcache = cached_model

On Feb 9, 2006, at 4:44 PM, Kian Wright wrote:
> Do you have to have all of your models derived from CachedModel and  
> running through the cache? Or can I do only selective ones  
> (hopefully)?
You can be selective if you like.
> I tried just switching one model over to CachedModel, and I get an  
> ''undefined class/module'' error on another model. But
that''s the
> User model, which is stored in the session. I didn''t think I was  
> caching sessions using memcached, but maybe I am...
You should check your configuration to see if you''re using memcached  
for sessions.
> /usr/lib/ruby/gems/1.8/gems/memcache-client-1.0.3/lib/memcache.rb: 
> 128:in `get''
> ...
Using dots removes critical information.
> /usr/lib/ruby/gems/1.8/gems/actionpack1.11.2/lib/action_controller/ 
> session/mem_cache_store.rb:59:in `restore''
> /usr/lib/ruby/1.8/cgi/session.rb:305:in `[]''
> /usr/lib/ruby/gems/1.8/gems/actionpack-1.11.2/lib/action_controller/ 
> cgi_process.rb:104:in `session''
> ...
> Looks like it is correctly finding the user, logging the user in,  
> and writing a last login time to the database. But then it  
> redirects and tries to pull the user object out of the session and  
> fails. Does CachedModel / memcache-client conflict with session  
> storage?
It depends.  You may have a version of cached_model with bugs.
> So maybe this is a session problem? Or is it having a problem  
> instantiating the User object pulled out of the cache when User is  
> derived from ActiveRecord::Base, not CachedModel?
Provide the missing information and I might be able to provide a  
solution.

PS: Use a relevant subject or use ''Re:'' if you''re
replying to something.

-- 
Eric Hodel - drbrain@segment7.net - http://segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com

Possibly Parallel Threads

Search for more possibly parallel threads

Rails - Jan 2006 - ActiveRecord + memcache = cached_model

[Rails] [ANN] ActiveRecord + memcache = cached_model

[Rails] [ANN] ActiveRecord + memcache = cached_model

[Rails] [ANN] ActiveRecord + memcache = cached_model

[Rails] [ANN] ActiveRecord + memcache = cached_model

[Rails] [ANN] ActiveRecord + memcache = cached_model

[Rails] [ANN] ActiveRecord + memcache = cached_model

[Rails] [ANN] ActiveRecord + memcache = cached_model

[Rails] [ANN] ActiveRecord + memcache = cached_model

[Rails] [ANN] ActiveRecord + memcache = cached_model

[Rails] [ANN] ActiveRecord + memcache = cached_model

[Rails] [ANN] ActiveRecord + memcache = cached_model

Possibly Parallel Threads