Hi all, I''m using profile.setFBML for an app I''m working on and in development I''m seeing fairly lengthy response times. It normally seems to return well within 1-3 seconds but occasionally seems to spike at around 20 seconds. As I''m calling setFBML in response to a user action (on a canvas page), this could give a really bad user experience. I don''t think my FBML pages are especially large so I''m wondering, are these sorts of timings normal? If so, is anyone already using an asynchronous queuing system to cope with this issue that they''d be willing to point me at? I could roll my own, but if setFBML typically takes a while it seems like something that would be useful for a large number of facebooker users. Regards, Lee. -- Lee Mallabone. Crossbone Systems Ltd. http://www.crossbonesystems.com/ http://www.fonicmonkey.net/ http://CambridgeWebHeads.ning.com/
We process all API calls through a set of starling queues. We have a
ProfileUpdate object so our flow looks like:
class User
def refresh_profile
ProfileUpdate.new(self.faceook_id).enqueue
end
end
We then have a job that pulls Profile Update messages off of the queue
and calls execute on them. The execute method does the profile update.
My using the command pattern we know we can throw any object the
implements execute onto the queue and the processing daemons will
handle it.
Mike
On Jan 13, 2009, at 9:53 AM, Lee Mallabone wrote:
> Hi all,
>
> I''m using profile.setFBML for an app I''m working on and
in development
> I''m seeing fairly lengthy response times. It normally seems to
return
> well within 1-3 seconds but occasionally seems to spike at around 20
> seconds.
>
> As I''m calling setFBML in response to a user action (on a canvas
> page), this could give a really bad user experience. I don''t think
my
> FBML pages are especially large so I''m wondering, are these sorts
of
> timings normal?
>
> If so, is anyone already using an asynchronous queuing system to cope
> with this issue that they''d be willing to point me at? I could
roll my
> own, but if setFBML typically takes a while it seems like something
> that would be useful for a large number of facebooker users.
>
> Regards,
> Lee.
>
>
> --
> Lee Mallabone.
> Crossbone Systems Ltd.
>
> http://www.crossbonesystems.com/
> http://www.fonicmonkey.net/
> http://CambridgeWebHeads.ning.com/
> _______________________________________________
> Facebooker-talk mailing list
> Facebooker-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/facebooker-talk
--
Mike Mangino
http://www.elevatedrails.com
Did you see Ryan Bs screencast on Starling and Workling? Its probably a suitable solution for "out-lining" API calls. Alan. Lee Mallabone wrote:> Hi all, > > I''m using profile.setFBML for an app I''m working on and in development > I''m seeing fairly lengthy response times. It normally seems to return > well within 1-3 seconds but occasionally seems to spike at around 20 > seconds. > > As I''m calling setFBML in response to a user action (on a canvas > page), this could give a really bad user experience. I don''t think my > FBML pages are especially large so I''m wondering, are these sorts of > timings normal? > > If so, is anyone already using an asynchronous queuing system to cope > with this issue that they''d be willing to point me at? I could roll my > own, but if setFBML typically takes a while it seems like something > that would be useful for a large number of facebooker users. > > Regards, > Lee. > >-- <a href="http://wikimediafoundation.org/wiki/Donate/en"><img border="0" alt="Wikipedia Affiliate Button" src="http://upload.wikimedia.org/wikipedia/foundation/5/58/2008_fundraiser_micro_button-en.png" /></a>
What data are you storing in the queue object? The Facebook sessions timeout after 1 hour unless extended permissions right? Does this mean that if the daemon doesn''t get to the queue within 30 minutes of the user''s last hit to your app it will fail? If this is true, then it would seem that you should keep multiple daemons running to keep the queue as low as possible. On Tue, Jan 13, 2009 at 9:05 AM, Mike Mangino <mmangino at elevatedrails.com> wrote:> We process all API calls through a set of starling queues. We have a > ProfileUpdate object so our flow looks like: > > class User > def refresh_profile > ProfileUpdate.new(self.faceook_id).enqueue > end > end > > We then have a job that pulls Profile Update messages off of the queue and > calls execute on them. The execute method does the profile update. > > My using the command pattern we know we can throw any object the implements > execute onto the queue and the processing daemons will handle it. > > Mike > > On Jan 13, 2009, at 9:53 AM, Lee Mallabone wrote: > >> Hi all, >> >> I''m using profile.setFBML for an app I''m working on and in development >> I''m seeing fairly lengthy response times. It normally seems to return >> well within 1-3 seconds but occasionally seems to spike at around 20 >> seconds. >> >> As I''m calling setFBML in response to a user action (on a canvas >> page), this could give a really bad user experience. I don''t think my >> FBML pages are especially large so I''m wondering, are these sorts of >> timings normal? >> >> If so, is anyone already using an asynchronous queuing system to cope >> with this issue that they''d be willing to point me at? I could roll my >> own, but if setFBML typically takes a while it seems like something >> that would be useful for a large number of facebooker users. >> >> Regards, >> Lee. >> >> >> -- >> Lee Mallabone. >> Crossbone Systems Ltd. >> >> http://www.crossbonesystems.com/ >> http://www.fonicmonkey.net/ >> http://CambridgeWebHeads.ning.com/ >> _______________________________________________ >> Facebooker-talk mailing list >> Facebooker-talk at rubyforge.org >> http://rubyforge.org/mailman/listinfo/facebooker-talk > > -- > Mike Mangino > http://www.elevatedrails.com > > > > _______________________________________________ > Facebooker-talk mailing list > Facebooker-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/facebooker-talk >
I just queued up my calls that set the profile FBML. That API call doesn''t need a session key so you only have to store the user_id and the FBML (or the data to generate the FBML) you intend to send. I''d be interested to hear other people''s approaches if they are indeed queuing all API calls. Lee. 2009/1/30 Jonathan Otto <jonathan.otto at gmail.com>:> What data are you storing in the queue object? > > The Facebook sessions timeout after 1 hour unless extended permissions > right? Does this mean that if the daemon doesn''t get to the queue > within 30 minutes of the user''s last hit to your app it will fail? > > If this is true, then it would seem that you should keep multiple > daemons running to keep the queue as low as possible. > > On Tue, Jan 13, 2009 at 9:05 AM, Mike Mangino > <mmangino at elevatedrails.com> wrote: >> We process all API calls through a set of starling queues. We have a >> ProfileUpdate object so our flow looks like: >> >> class User >> def refresh_profile >> ProfileUpdate.new(self.faceook_id).enqueue >> end >> end >> >> We then have a job that pulls Profile Update messages off of the queue and >> calls execute on them. The execute method does the profile update. >> >> My using the command pattern we know we can throw any object the implements >> execute onto the queue and the processing daemons will handle it. >> >> Mike >> >> On Jan 13, 2009, at 9:53 AM, Lee Mallabone wrote: >> >>> Hi all, >>> >>> I''m using profile.setFBML for an app I''m working on and in development >>> I''m seeing fairly lengthy response times. It normally seems to return >>> well within 1-3 seconds but occasionally seems to spike at around 20 >>> seconds. >>> >>> As I''m calling setFBML in response to a user action (on a canvas >>> page), this could give a really bad user experience. I don''t think my >>> FBML pages are especially large so I''m wondering, are these sorts of >>> timings normal? >>> >>> If so, is anyone already using an asynchronous queuing system to cope >>> with this issue that they''d be willing to point me at? I could roll my >>> own, but if setFBML typically takes a while it seems like something >>> that would be useful for a large number of facebooker users. >>> >>> Regards, >>> Lee. >>> >>> >>> -- >>> Lee Mallabone. >>> Crossbone Systems Ltd. >>> >>> http://www.crossbonesystems.com/ >>> http://www.fonicmonkey.net/ >>> http://CambridgeWebHeads.ning.com/ >>> _______________________________________________ >>> Facebooker-talk mailing list >>> Facebooker-talk at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/facebooker-talk >> >> -- >> Mike Mangino >> http://www.elevatedrails.com >> >> >> >> _______________________________________________ >> Facebooker-talk mailing list >> Facebooker-talk at rubyforge.org >> http://rubyforge.org/mailman/listinfo/facebooker-talk >> >-- Lee Mallabone. Crossbone Systems Ltd. Tired of hunting for great people? Post jobs on your facebook profile with Head Hunting: http://apps.facebook.com/headhunting/pages/recruiter
On Jan 30, 2009, at 12:48 AM, Jonathan Otto wrote:> What data are you storing in the queue object?We have two different types. One type just stores the minimum data necessary to pull the data it needs from the DB. These are the profile update type messages that need a lot of data to do their job. The objects that send messages store everything they need to send the message.> > > The Facebook sessions timeout after 1 hour unless extended permissions > right? Does this mean that if the daemon doesn''t get to the queue > within 30 minutes of the user''s last hit to your app it will fail?That''s right, but we want the queues to process quickly anyway. In some of our apps, users will interact with the application and then go view a user''s profile to look for the change. We try to keep our queue under 100 messages all of the time. We have 2 machines dedicated to just processing these API calls. We run about 2 million messages through the queues per day. Then again, these applications have a lot of users :) Mike> > > If this is true, then it would seem that you should keep multiple > daemons running to keep the queue as low as possible. >We''re currently running about 15 processes per machine.> On Tue, Jan 13, 2009 at 9:05 AM, Mike Mangino > <mmangino at elevatedrails.com> wrote: >> We process all API calls through a set of starling queues. We have a >> ProfileUpdate object so our flow looks like: >> >> class User >> def refresh_profile >> ProfileUpdate.new(self.faceook_id).enqueue >> end >> end >> >> We then have a job that pulls Profile Update messages off of the >> queue and >> calls execute on them. The execute method does the profile update. >> >> My using the command pattern we know we can throw any object the >> implements >> execute onto the queue and the processing daemons will handle it. >> >> Mike >> >> On Jan 13, 2009, at 9:53 AM, Lee Mallabone wrote: >> >>> Hi all, >>> >>> I''m using profile.setFBML for an app I''m working on and in >>> development >>> I''m seeing fairly lengthy response times. It normally seems to >>> return >>> well within 1-3 seconds but occasionally seems to spike at around 20 >>> seconds. >>> >>> As I''m calling setFBML in response to a user action (on a canvas >>> page), this could give a really bad user experience. I don''t think >>> my >>> FBML pages are especially large so I''m wondering, are these sorts of >>> timings normal? >>> >>> If so, is anyone already using an asynchronous queuing system to >>> cope >>> with this issue that they''d be willing to point me at? I could >>> roll my >>> own, but if setFBML typically takes a while it seems like something >>> that would be useful for a large number of facebooker users. >>> >>> Regards, >>> Lee. >>> >>> >>> -- >>> Lee Mallabone. >>> Crossbone Systems Ltd. >>> >>> http://www.crossbonesystems.com/ >>> http://www.fonicmonkey.net/ >>> http://CambridgeWebHeads.ning.com/ >>> _______________________________________________ >>> Facebooker-talk mailing list >>> Facebooker-talk at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/facebooker-talk >> >> -- >> Mike Mangino >> http://www.elevatedrails.com >> >> >> >> _______________________________________________ >> Facebooker-talk mailing list >> Facebooker-talk at rubyforge.org >> http://rubyforge.org/mailman/listinfo/facebooker-talk >>-- Mike Mangino http://www.elevatedrails.com
For anyone else thinking of implementing a queue, this seems to work pretty well and lets you implement a light weight daemon that doesn''t load up the Rails env since the message contains the URL and params. http://gist.github.com/55742 The FacebookQueue class simply abstracts any queue system you might use. Let me know if you see any problems with it. On Fri, Jan 30, 2009 at 7:53 AM, Mike Mangino <mmangino at elevatedrails.com> wrote:> > On Jan 30, 2009, at 12:48 AM, Jonathan Otto wrote: > >> What data are you storing in the queue object? > > We have two different types. One type just stores the minimum data necessary > to pull the data it needs from the DB. These are the profile update type > messages that need a lot of data to do their job. The objects that send > messages store everything they need to send the message. > >> >> >> The Facebook sessions timeout after 1 hour unless extended permissions >> right? Does this mean that if the daemon doesn''t get to the queue >> within 30 minutes of the user''s last hit to your app it will fail? > > That''s right, but we want the queues to process quickly anyway. In some of > our apps, users will interact with the application and then go view a user''s > profile to look for the change. We try to keep our queue under 100 messages > all of the time. > > We have 2 machines dedicated to just processing these API calls. We run > about 2 million messages through the queues per day. > > Then again, these applications have a lot of users :) > > Mike > >> >> >> If this is true, then it would seem that you should keep multiple >> daemons running to keep the queue as low as possible. >> > > We''re currently running about 15 processes per machine. > >> On Tue, Jan 13, 2009 at 9:05 AM, Mike Mangino >> <mmangino at elevatedrails.com> wrote: >>> >>> We process all API calls through a set of starling queues. We have a >>> ProfileUpdate object so our flow looks like: >>> >>> class User >>> def refresh_profile >>> ProfileUpdate.new(self.faceook_id).enqueue >>> end >>> end >>> >>> We then have a job that pulls Profile Update messages off of the queue >>> and >>> calls execute on them. The execute method does the profile update. >>> >>> My using the command pattern we know we can throw any object the >>> implements >>> execute onto the queue and the processing daemons will handle it. >>> >>> Mike >>> >>> On Jan 13, 2009, at 9:53 AM, Lee Mallabone wrote: >>> >>>> Hi all, >>>> >>>> I''m using profile.setFBML for an app I''m working on and in development >>>> I''m seeing fairly lengthy response times. It normally seems to return >>>> well within 1-3 seconds but occasionally seems to spike at around 20 >>>> seconds. >>>> >>>> As I''m calling setFBML in response to a user action (on a canvas >>>> page), this could give a really bad user experience. I don''t think my >>>> FBML pages are especially large so I''m wondering, are these sorts of >>>> timings normal? >>>> >>>> If so, is anyone already using an asynchronous queuing system to cope >>>> with this issue that they''d be willing to point me at? I could roll my >>>> own, but if setFBML typically takes a while it seems like something >>>> that would be useful for a large number of facebooker users. >>>> >>>> Regards, >>>> Lee. >>>> >>>> >>>> -- >>>> Lee Mallabone. >>>> Crossbone Systems Ltd. >>>> >>>> http://www.crossbonesystems.com/ >>>> http://www.fonicmonkey.net/ >>>> http://CambridgeWebHeads.ning.com/ >>>> _______________________________________________ >>>> Facebooker-talk mailing list >>>> Facebooker-talk at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/facebooker-talk >>> >>> -- >>> Mike Mangino >>> http://www.elevatedrails.com >>> >>> >>> >>> _______________________________________________ >>> Facebooker-talk mailing list >>> Facebooker-talk at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/facebooker-talk >>> > > -- > Mike Mangino > http://www.elevatedrails.com > > > >