Max Williams
2008-Nov-16 10:46 UTC
Farming off zipping task to backgroundrb - advice requested!
We have a problem at the moment: users build up lessons, which are then zipped up with the lesson player into a standalone executable. This has always been ok for us but the lessons and the player just got a lot bigger, and the zipping process is now hogging resources too much: we have an nginx server with a cluster of 12 mongrels, and if 3 people zip a lesson at once, between them the three mongrels use up 100% of the cpu time and tend to zombify. What i''d like to do is move the zipping out of the mongrels into seperate process, using backgroundrb. (I''ve got backgroundrb working fine with some other stuff and am happy with it generally). This will mean that the zips are built in a queue instead of all at once, thus avoiding meltdown. It will also stop mongrels getting out of control. My plan is this - does it sound sensible? Controller action 1 - ''build_zip''. Call backgroundrb and tell it to make a zip of this lesson. Fall through to ''build_zip'' view, which just says "building zip file". Backgroundrb - builds zip. When it''s finished, puts an entry in a table called ''built_lesson_zips'' Back on ''build_zip'' view page: javascript action triggered by page load: polls (via built_lesson_zips_controller) the built_lesson_zips database table to see if the relevant zip is there. If not, refresh the ''building zip file'' message (or do something so that the user can see it hasn''t just crashed), wait 5 seconds and try again. If it is there, then call controller action 2, ''built_zip''. Controller action 2 - ''built_zip''. Proceed as before - start downloading the zip to the user etc. Any feedback, suggestions, etc? Polling the table every five seconds using javascript feels a bit dirty but since there''s 12 mongrels and only one backgroundrb, getting backgroundrb to make a callback to the right mongrel seemed too difficult for my limited knowledge, so i wanted to keep the communication purely one-way. -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---