Rails - Dec 2009 - how to manage the caching of the results of long running queries

Greetings:

I have a rails app with a dashboard page. I''m running under Apache/
Passenger. The dashboard page has several queries run via a dashboard
() action on a controller. The queries can take 30-60 seconds to
complete before the dashboard page is rendered. I''m using cache_page()
in the controller and the rendered page is cached in public, with
subsequent requests satisfied from cache. This is all fine.

The problem I have is refreshing the cache after it''s invalidated
(delete files from public via cron). All clients that call the
dashboard action while the page cache is empty will cause queries to
be run against the database. That is, of course, the kind of thing I''m
trying to avoid by using cache_page().

What are my options?

Thanks.

--

You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

Depends on how many queries they are and how long they take
individually. If it''s one query that takes 30 seconds, then look at
breaking it up or optimizing it. If it''s several queries that take a
few seconds each, look into caching their results with
Rails.cache.fetch. Then if a request hits while the first page is 15
seconds into building, it can at least take advantage of those 15
seconds, and only duplicate the last 15.

On Dec 2, 4:00 pm, gallivan
<jay.galli...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
wrote:
> Greetings:
>
> I have a rails app with a dashboard page. I''m running under
Apache/
> Passenger. The dashboard page has several queries run via a dashboard
> () action on a controller. The queries can take 30-60 seconds to
> complete before the dashboard page is rendered. I''m using
cache_page()
> in the controller and the rendered page is cached in public, with
> subsequent requests satisfied from cache. This is all fine.
>
> The problem I have is refreshing the cache after it''s invalidated
> (delete files from public via cron). All clients that call the
> dashboard action while the page cache is empty will cause queries to
> be run against the database. That is, of course, the kind of thing
I''m
> trying to avoid by using cache_page().
>
> What are my options?
>
> Thanks.
--

You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

This sounds like fragment caching rather than page caching. Is that right?

On Wed, Dec 2, 2009 at 10:11 PM, David
<furberd-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> Depends on how many queries they are and how long they take
> individually. If it''s one query that takes 30 seconds, then look
at
> breaking it up or optimizing it. If it''s several queries that take
a
> few seconds each, look into caching their results with
> Rails.cache.fetch. Then if a request hits while the first page is 15
> seconds into building, it can at least take advantage of those 15
> seconds, and only duplicate the last 15.
>
> On Dec 2, 4:00 pm, gallivan
<jay.galli...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > Greetings:
> >
> > I have a rails app with a dashboard page. I''m running under
Apache/
> > Passenger. The dashboard page has several queries run via a dashboard
> > () action on a controller. The queries can take 30-60 seconds to
> > complete before the dashboard page is rendered. I''m using
cache_page()
> > in the controller and the rendered page is cached in public, with
> > subsequent requests satisfied from cache. This is all fine.
> >
> > The problem I have is refreshing the cache after it''s
invalidated
> > (delete files from public via cron). All clients that call the
> > dashboard action while the page cache is empty will cause queries to
> > be run against the database. That is, of course, the kind of thing
I''m
> > trying to avoid by using cache_page().
> >
> > What are my options?
> >
> > Thanks.
>
> --
>
> You received this message because you are subscribed to the Google Groups
> "Ruby on Rails: Talk" group.
> To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To unsubscribe from this group, send email to
>
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org<rubyonrails-talk%2Bunsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>
> .
> For more options, visit this group at
> http://groups.google.com/group/rubyonrails-talk?hl=en.
>
>
>
--

You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

You could do that, but if the long running queries are executed in the
controller before you get to rendering (where fragment caching is
done), then it won''t do much good against the problem you are trying
to solve, because it will execute the queries for 30 seconds before
getting to the rendering.

@projects = Rails.cache.fetch("user#{current_user.id}_projects")
{ current_user.projects.with_long_running_query }
@tasks = Rails.cache.fetch("user#{current_user.id}_tasks")
{ @projects.map {|project|
project.tasks.with_some_special_conditions }

Or something like that...

On Dec 3, 12:03 pm, jay gallivan
<jay.galli...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
wrote:
> This sounds like fragment caching rather than page caching. Is that right?
>
>
>
> On Wed, Dec 2, 2009 at 10:11 PM, David
<furb...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > Depends on how many queries they are and how long they take
> > individually. If it''s one query that takes 30 seconds, then
look at
> > breaking it up or optimizing it. If it''s several queries that
take a
> > few seconds each, look into caching their results with
> > Rails.cache.fetch. Then if a request hits while the first page is 15
> > seconds into building, it can at least take advantage of those 15
> > seconds, and only duplicate the last 15.
>
> > On Dec 2, 4:00 pm, gallivan
<jay.galli...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > > Greetings:
>
> > > I have a rails app with a dashboard page. I''m running
under Apache/
> > > Passenger. The dashboard page has several queries run via a
dashboard
> > > () action on a controller. The queries can take 30-60 seconds to
> > > complete before the dashboard page is rendered. I''m
using cache_page()
> > > in the controller and the rendered page is cached in public, with
> > > subsequent requests satisfied from cache. This is all fine.
>
> > > The problem I have is refreshing the cache after it''s
invalidated
> > > (delete files from public via cron). All clients that call the
> > > dashboard action while the page cache is empty will cause queries
to
> > > be run against the database. That is, of course, the kind of
thing I''m
> > > trying to avoid by using cache_page().
>
> > > What are my options?
>
> > > Thanks.
>
> > --
>
> > You received this message because you are subscribed to the Google
Groups
> > "Ruby on Rails: Talk" group.
> > To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> > To unsubscribe from this group, send email to
> >
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org<rubyonrails-talk%2Bunsubscrib
e@googlegroups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/rubyonrails-talk?hl=en.
--

You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.