Rails - Nov 2006 - Caching application data

Hi,

say my application has a small 20 row information table in the database 
that stores names of airlines:
"continental", "delta", united" , etc...

These names will be used in various pages of the application for all the 
website users for different actions. They do not expect to change often.

what I do now is - when a user logs in - these airline names are queried 
from the db and stored in session. Then every action that needs them 
will get them from session instead of querying the database.

What I wanted to do is not query this table again and again for every 
action. It just occured to me that i merely replaced a db query for a 
session file access. Maybe a little faster in some cases for also not 
the best way to go.

What''s the best solution for this? can I store this array of strings in
some global constant after fetching them from the database for the first 
time when a user logs in? how? what will happen when I change the 
contents of the table in the database every few weeks? (i guess i still 
have this problem with sessions too, if a user is logged in while I make 
the change)

Thanks!

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Serialization in either a YAML file or maybe as ruby source code that
can be eval''d seem to be the most obvious solutions to that problem, to
me.  Another idea that you might try is putting all that data in a
value-object class (not unlike TimeZone).


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Alon wrote:
> Hi,
> 
> say my application has a small 20 row information table in the database 
> that stores names of airlines:
> "continental", "delta", united" , etc...
> 
> These names will be used in various pages of the application for all the 
> website users for different actions. They do not expect to change often.
> 
> what I do now is - when a user logs in - these airline names are queried 
> from the db and stored in session. Then every action that needs them 
> will get them from session instead of querying the database.
> 
> What I wanted to do is not query this table again and again for every 
> action. It just occured to me that i merely replaced a db query for a 
> session file access. Maybe a little faster in some cases for also not 
> the best way to go.
> 
> What''s the best solution for this? can I store this array of
strings in
> some global constant after fetching them from the database for the first 
> time when a user logs in? how? what will happen when I change the 
> contents of the table in the database every few weeks? (i guess i still 
> have this problem with sessions too, if a user is logged in while I make 
> the change)
> 
> Thanks!
You could use class methods accessors to a class variable.

Simply fetch your data from the DB if the class variable is nil, and use 
the var if it has a value.  The trick is to reset the cached value to 
nil after any change that might affect it.

  class Airlines
    def self.all_airlines
      @@all_airlines ||= Airlines.find(:all)
    end
    def self.expire_airlines
      @@all_airlines = nil
    end
  end

  Airlines.all_airlines # calls DB
  Airlines.all_airlines # uses already fetched @@all_airlined
  Airlines.expire_airlines
  Airlines.all_airlines # calls DB

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Alex,

thanks a bunch! makes a lot of sense. Will use your suggestion.

Alon.

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Hmm... seems to be querying the database every single time for some 
reason...

this is how it looks like (categories instead of airlines in this 
example)

class Category < ActiveRecord::Base

  def self.get_categories
    @@all_categories ||= find(:all, :order => "name")
  end

  def self.expire_categories
    @@all_categories = nil
  end
end

# in some controller
@cat = Category.get_categories # this statement always ends up in a db 
query

expire_categories is not used yet in my code so nobody is setting it to 
nil

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Alon wrote:
> Hmm... seems to be querying the database every single time for some
> reason...
>
> this is how it looks like (categories instead of airlines in this
> example)
>
> class Category < ActiveRecord::Base
>
>   def self.get_categories
>     @@all_categories ||= find(:all, :order => "name")
>   end
>
>   def self.expire_categories
>     @@all_categories = nil
>   end
> end
>
> # in some controller
> @cat = Category.get_categories # this statement always ends up in a db
> query
>
> expire_categories is not used yet in my code so nobody is setting it to
> nil
>
> --
> Posted via http://www.ruby-forum.com/.
In development mode, the models get reloaded on each request (IIRC), so
the value of @@all_categories gets reset every time you call the page.

_Kevin


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Alon wrote:
> Hmm... seems to be querying the database every single time for some 
> reason...
> 
> this is how it looks like (categories instead of airlines in this 
> example)
> 
> class Category < ActiveRecord::Base
> 
>   def self.get_categories
>     @@all_categories ||= find(:all, :order => "name")
>   end
> 
>   def self.expire_categories
>     @@all_categories = nil
>   end
> end
> 
> # in some controller
> @cat = Category.get_categories # this statement always ends up in a db 
> query
> 
> expire_categories is not used yet in my code so nobody is setting it to 
> nil

I think the best thing to do is at the start of each day, the first 
person to access the database dumps that information into an xml file. 
From then on you only need to read the file.

This approch will limit db access quite a bit.

You might want to name the file after todays date so if the file does 
not exist, then we can assume its time for a new query.

This may not seem like an elegant approach but it is when you consider 
the load your placing elsewhere.



-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

CatLady [] wrote:
> I think the best thing to do is at the start of each day, the first 
> person to access the database dumps that information into an xml file. 
> From then on you only need to read the file.
> 
> This approch will limit db access quite a bit.
> 
> You might want to name the file after todays date so if the file does 
> not exist, then we can assume its time for a new query.
> 
> This may not seem like an elegant approach but it is when you consider 
> the load your placing elsewhere.
I don''t know if I would recommend reading from the file system as a 
better approach to reading from the DB. If you are going to dump into a 
file I would suggest YAML, it''s extremely easy to marshall/unmarshall 
ruby objects into/out of it. It''s all built in, nice and easy. Plus
it''s
more lightweight.

With that said have you thought of using something like Memcached? It 
was built for exactly this purpose. It''s extremely efficient, and has 
been proven successful time and time again. It''s extremely to hook up
to
ruby/rails as well. I use it everyday and swear by it.

If you can''t hook up Memcached, then you could create a simple
Singleton
class and cache the data in there. Pretty simple and straightforward, 
and it works. Very lightweight.

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Thanks catlady and Mark - very good suggestions and very appreciated.
> If you can''t hook up Memcached, then you could create a simple
Singleton
> class and cache the data in there. Pretty simple and straightforward, 
> and it works. Very lightweight.
Yes, singleton class is my preference actually, as suggested by Alex. 
However, as I indicated in my last posting above, it doesn''t seem to 
work for me - the application works but it seems to be accessing the 
database every single time for the data... I am probably doing something 
wrong - I''ll appreciate if somebody can see if it''s something
obvious
(short code snipet above)

Alon.

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Use memcached, dump it onto that. Really fast.

Vish

On 11/6/06, Alon
<rails-mailing-list-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org>
wrote:
>
> Thanks catlady and Mark - very good suggestions and very appreciated.
>
> > If you can''t hook up Memcached, then you could create a
simple Singleton
> > class and cache the data in there. Pretty simple and straightforward,
> > and it works. Very lightweight.
>
> Yes, singleton class is my preference actually, as suggested by Alex.
> However, as I indicated in my last posting above, it doesn''t seem
to
> work for me - the application works but it seems to be accessing the
> database every single time for the data... I am probably doing something
> wrong - I''ll appreciate if somebody can see if it''s
something obvious
> (short code snipet above)
>
> Alon.
>
> --
> Posted via http://www.ruby-forum.com/.
>
> >
>
--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Alon wrote:
> Thanks catlady and Mark - very good suggestions and very appreciated.
> Yes, singleton class is my preference actually, as suggested by Alex. 
> However, as I indicated in my last posting above, it doesn''t seem
to
> work for me - the application works but it seems to be accessing the 
> database every single time for the data... I am probably doing something 
> wrong - I''ll appreciate if somebody can see if it''s
something obvious
> (short code snipet above)
> 
> Alon.
The reason it wasn''t working for you was because you weren''t
using a
singleton. Here''s a very very simple singleton cache

require ''singleton''
class Cache
  include Singleton
  attr_reader :cache

  def initialize
    @cache = {}
  end

  def get(key)
    self.cache[key.to_s.downcase]
  end

  def put(key, value)
    self.cache[key.to_s.downcase] = value
    value
  end

  def delete (key)
    self.cache.delete(key.to_s.downcase)
  end

  def reset
    self.cache.clear
  end

end

c = Cache.instance
puts c.get(:foo)
c.put(:foo, "hello")
3.times { puts c.get(:foo) }
c.delete(:foo)
puts c.get(:foo)

If you run that you''ll see when it does the loop it keeps returning 
"hello" during the loop block. You can start getting clever with this 
sort of thing and have the get method yield to a block if it can''t find
the key, etc... Anyway, you get the idea.

Also, I don''t know if you were running in development mode, but if you 
were remember that AR models and controllers get reloaded ever single 
time you make a request. So if you were expecting them to hold some sort 
of state, apart from sessions in the controller, then that might be part 
of your problem as well.

What I''ve outlined briefly above will hold state for the entire length 
of your application. You can''t kill a singleton until you kill the ruby
process.

Anyway, hope this helps.


-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Thanks again Mark.
> Also, I don''t know if you were running in development mode, but if
you
> were remember that AR models and controllers get reloaded ever single 
> time you make a request. So if you were expecting them to hold some sort 
> of state, apart from sessions in the controller, then that might be part 
> of your problem as well.
That''s exactly what happened. I actually suspected it to be the case, 
however for some reason in the past my models were not getting loaded 
every time in development mode, but only the views and controllers. I 
discovered this because most changes I was making to my model classes 
were not taking effect unless I restarted webricks. I actually just saw 
some people''s posts about it and somebody included a link to an erratta
for David''s Agile book with an indication of this bug.

Having said that - it appear to have been loaded every single time in 
this case, which confuses me, but most importantly my problem is solved 
so thanks a lot to everybody!

Alon.

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

On Nov 6, 2:53 pm, Alon
<rails-mailing-l...-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org>
wrote:
> Thanks again Mark.
>
> > Also, I don''t know if you were running in development mode,
but if you
> > were remember that AR models and controllers get reloaded ever single
> > time you make a request. So if you were expecting them to hold some
sort
> > of state, apart from sessions in the controller, then that might be
part
> > of your problem as well.That''s exactly what happened. I
actually suspected it to be the case,
> however for some reason in the past my models were not getting loaded
> every time in development mode, but only the views and controllers. I
> discovered this because most changes I was making to my model classes
> were not taking effect unless I restarted webricks. I actually just saw
> some people''s posts about it and somebody included a link to an
erratta
> for David''s Agile book with an indication of this bug.
>
> Having said that - it appear to have been loaded every single time in
> this case, which confuses me, but most importantly my problem is solved
> so thanks a lot to everybody!
> 
> Alon.
> 
> --
> Posted viahttp://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Hi~

On Nov 6, 2006, at 5:37 AM, Mark Bates wrote:
>
> Alon wrote:
>> Thanks catlady and Mark - very good suggestions and very appreciated.
>> Yes, singleton class is my preference actually, as suggested by Alex.
>> However, as I indicated in my last posting above, it doesn''t
seem to
>> work for me - the application works but it seems to be accessing the
>> database every single time for the data... I am probably doing  
>> something
>> wrong - I''ll appreciate if somebody can see if it''s
something obvious
>> (short code snipet above)
>>
>> Alon.
>
> The reason it wasn''t working for you was because you
weren''t using a
> singleton. Here''s a very very simple singleton cache
>
> require ''singleton''
> class Cache
>   include Singleton
>   attr_reader :cache
>
>   def initialize
>     @cache = {}
>   end
>
>   def get(key)
>     self.cache[key.to_s.downcase]
>   end
>
>   def put(key, value)
>     self.cache[key.to_s.downcase] = value
>     value
>   end
>
>   def delete (key)
>     self.cache.delete(key.to_s.downcase)
>   end
>
>   def reset
>     self.cache.clear
>   end
>
> end
>
> c = Cache.instance
> puts c.get(:foo)
> c.put(:foo, "hello")
> 3.times { puts c.get(:foo) }
> c.delete(:foo)
> puts c.get(:foo)
>
> If you run that you''ll see when it does the loop it keeps
returning
> "hello" during the loop block. You can start getting clever with
this
> sort of thing and have the get method yield to a block if it can''t
> find
> the key, etc... Anyway, you get the idea.
>
> Also, I don''t know if you were running in development mode, but if
you
> were remember that AR models and controllers get reloaded ever single
> time you make a request. So if you were expecting them to hold some  
> sort
> of state, apart from sessions in the controller, then that might be  
> part
> of your problem as well.
>
> What I''ve outlined briefly above will hold state for the entire
length
> of your application. You can''t kill a singleton until you kill the
> ruby
> process.
>
> Anyway, hope this helps.
	While this may seem like the solution it is not. This will not work  
when you have multiple mongrels or fcgi''s running for your app.  
Because each process will have its own copy of the singleton cache  
and your users are not garaunteed to always hit the same backend on  
each request> So the data in the cache will be inconsistent depending  
on which backend serves a certain request.

	The only way to do this correctly is to make it a drb server or just  
use memcached. The singleton cache stuff is great if you can  
garauntee that your app will never use more then one backend. But  
that will not scale.

Cheers-

-- Ezra Zygmuntowicz 
-- Lead Rails Evangelist
-- ez-NLltGlunAUd/unjJdyJNww@public.gmane.org
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)



--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---