Rails - Nov 2006 - Concurrency problem with memcached fragments mongrel cluster

Hi,

I have the following question:  What happens when you memcache fragments
and you are running with a mongrel cluster.  In the case when we check
read_fragment() in our action/controller and by the time Rails rechecks
to see if the fragment exists in the view, the fragment could have
possibly expired in memcache, or could have gotten deleted if you are
running some expiration sweeper of some sort.  This means that none of
the instance variables or SQL was run since read_fragment() returned
true saying there is a fragment.  Does this mean view rendering will
crash because it would try to create the cache block?  How does one fix
this window if this is a problem?  How do multiple mongrels synchronize
around this shared fragment?

Thanks.

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

> I have the following question:  What happens when you memcache fragments
> and you are running with a mongrel cluster.  In the case when we check
> read_fragment() in our action/controller and by the time Rails rechecks
> to see if the fragment exists in the view, the fragment could have
> possibly expired in memcache, or could have gotten deleted if you are
> running some expiration sweeper of some sort.  This means that none of
> the instance variables or SQL was run since read_fragment() returned
> true saying there is a fragment.  Does this mean view rendering will
> crash because it would try to create the cache block?  How does one fix
> this window if this is a problem?  How do multiple mongrels synchronize
> around this shared fragment?
This could very well happen, but if your memcache ram size is large enough 
and you''ve told it not to expire, probably won''t.

But...memcache is *not* permanent storage... they make that very clear. 
So if you''re relying on it being there long term, well...
don''t.

:)

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Yes this could happen as well, but if your cache size is large enough,
it''ll
happen probably about one in a million times. Can''t think of a way
around
this.

Vish

On 11/28/06, Philip Hallstrom
<rails-SUcgGwS4C16SUMMaM/qcSw@public.gmane.org>
wrote:
>
>
> > I have the following question:  What happens when you memcache
fragments
> > and you are running with a mongrel cluster.  In the case when we check
> > read_fragment() in our action/controller and by the time Rails
rechecks
> > to see if the fragment exists in the view, the fragment could have
> > possibly expired in memcache, or could have gotten deleted if you are
> > running some expiration sweeper of some sort.  This means that none of
> > the instance variables or SQL was run since read_fragment() returned
> > true saying there is a fragment.  Does this mean view rendering will
> > crash because it would try to create the cache block?  How does one
fix
> > this window if this is a problem?  How do multiple mongrels
synchronize
> > around this shared fragment?
>
> This could very well happen, but if your memcache ram size is large enough
> and you''ve told it not to expire, probably won''t.
>
> But...memcache is *not* permanent storage... they make that very clear.
> So if you''re relying on it being there long term, well...
don''t.
>
> :)
>
> >
>

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

My understanding is that memcached uses LRU, so calling read_fragment() will
push the fragment to the top of he used list.  This should give it a good
chance of surviving until you use it in the view.  However, you should check
out extended_fragment_cache [1].  Though I have never used it, I believe
when calling read_fragment(), it saves the cached fragment in an in-process
cache.  This would in theory cut down on the number of cache queries, and
possibly guarantee that the fragment exists (in the in-process cache) when
you hit the view.  I found this extension through Tyler Kovaks'' slides
titled "Efficient Memcached Usage" [2] (check out slide 8/11).

[1]
http://www.gemjack.com/gems/extended_fragment_cache-0.0.1/files/home/tony/projects/gemjack/gems/tmp/lib/extended_fragment_cache_rb.html
[2] http://sparecycles.org/efficient_memcached_usage/index.html

On 11/28/06, Vishnu Gopal
<g.vishnu-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
wrote:
>
> Yes this could happen as well, but if your cache size is large enough,
> it''ll happen probably about one in a million times. Can''t
think of a way
> around this.
>
> Vish
>
> On 11/28/06, Philip Hallstrom
<rails-SUcgGwS4C16SUMMaM/qcSw@public.gmane.org> wrote:
> >
> >
> > > I have the following question:  What happens when you memcache
> > fragments
> > > and you are running with a mongrel cluster.  In the case when we
check
> > > read_fragment() in our action/controller and by the time Rails
> > rechecks
> > > to see if the fragment exists in the view, the fragment could
have
> > > possibly expired in memcache, or could have gotten deleted if you
are
> > > running some expiration sweeper of some sort.  This means that
none of
> >
> > > the instance variables or SQL was run since read_fragment()
returned
> > > true saying there is a fragment.  Does this mean view rendering
will
> > > crash because it would try to create the cache block?  How does
one
> > fix
> > > this window if this is a problem?  How do multiple mongrels
> > synchronize
> > > around this shared fragment?
> >
> > This could very well happen, but if your memcache ram size is large
> > enough
> > and you''ve told it not to expire, probably won''t.
> >
> > But...memcache is *not* permanent storage... they make that very
clear.
> > So if you''re relying on it being there long term, well...
don''t.
> >
> > :)
> >
> >
> > > >
> >

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---