If I''m going to run a task that may take a long time, how do I start a task and return without waiting for the task to finish? Thanks. -- Jack Christensen jackc-/SOt/BrQZzOj3I+7jmQ39gC/G2K4zDHf@public.gmane.org
Jack Christensen wrote:> If I''m going to run a task that may take a long time, how do I start a > task and return without waiting for the task to finish? > > Thanks. >From my limited experience so far I don''t think Rails has this type of functionality. Lets assume I am correct, so how would I approach this? I would have a seperate ruby application (outside of your Rails app) that would listen for soap requests. I would probably start with have 2 different requests, a submit task request and get task status request (returns percent complete). Basically in a nutshell this other ruby application would be a "task processing server". You could then use ajax to poll for the status of a task to determine if it has been completed or not. I have done something like this in Java and it worked out well. In my case I built a photo album generator. When the user would request to "build" the photo album I would submit a request to the "taskserver". Every few second I would ask the "taskserver" for the percent complete and then display the percent complete on a web page. Anybody else have an idea? If not I would be to work with you on an implementation strategy. -- Scott F. Walter Scott F. Walter Principal Consultant Vivare, Inc. E: scott.walter-uosFFu51klvQT0dZR+AlfA@public.gmane.org E: scott-APWf0AbNa2kIjDr1QQGPvw@public.gmane.org Visit scottwalter.com <http://scottwalter.com> --Point. Click. Explore!
On 8/25/05, Scott F. Walter <scott.walter-uosFFu51klvQT0dZR+AlfA@public.gmane.org> wrote:> From my limited experience so far I don''t think Rails has this type of > functionality. Lets assume I am correct, so how would I approach this?um... def thread thread = Thread.new do while true rand(1000)*rand(1000)*rand(1000) end end render_text "thread going in background" end that what you''re after? (just put the code to execute in the thread block) Ben
I should also add that it''ll only work with FastCGI or Webrick - not plain old CGI. Ben
and also - don''t run that example code as is unless you want to choke up your CPU - and *especially* not on a shared hosting account ;-) just put your own code in there first, the while true... stuff is just an example. On 8/25/05, Ben Myles <ben.myles-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> I should also add that it''ll only work with FastCGI or Webrick - not > plain old CGI. > > Ben >
Ben Myles wrote:>On 8/25/05, Scott F. Walter <scott.walter-uosFFu51klvQT0dZR+AlfA@public.gmane.org> wrote: > > >> From my limited experience so far I don''t think Rails has this type of >>functionality. Lets assume I am correct, so how would I approach this? >> >> > >um... > >def thread > thread = Thread.new do > while true > rand(1000)*rand(1000)*rand(1000) > end > end > render_text "thread going in background" >end > >that what you''re after? (just put the code to execute in the thread block) > >Ben >_______________________________________________ >Rails mailing list >Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org >http://lists.rubyonrails.org/mailman/listinfo/rails > > >Alot simpler than I first thought. So how do you let the user know the process is done? I guess you could have some sort of Singleton-design pattern based class that you could use to hold the status of the long running process. That way you have something to poll to let the user know the process is still going on and didn''t abruptly fail. scott. -- Scott F. Walter Scott F. Walter Principal Consultant Vivare, Inc. E: scott.walter-uosFFu51klvQT0dZR+AlfA@public.gmane.org E: scott-APWf0AbNa2kIjDr1QQGPvw@public.gmane.org Visit scottwalter.com <http://scottwalter.com> --Point. Click. Explore!
On 8/25/05, Scott F. Walter <scott.walter-uosFFu51klvQT0dZR+AlfA@public.gmane.org> wrote:> Alot simpler than I first thought. So how do you let the user know the > process is done? I guess you could have some sort of Singleton-design > pattern based class that you could use to hold the status of the long > running process. That way you have something to poll to let the user > know the process is still going on and didn''t abruptly fail.Yep, I think that''d work fine. You could also make use of the Ajax features so that some kind of status indicator is updated in the background without the need for reloads. Just remember that everything in Rails is just pure Ruby code - if you can do it in Ruby you can do it in Rails. And if you''re using FastCGI or Webrick then your ruby process sticks around rather than starting again for every request, so you can do things like Thread.new. Ben
How about using Ajax to initiate it, and then monitoring a disk-file or database record that the long-running task updates with a periodic- update ajax call? Julian. On 25/08/2005, at 8:35 AM, Scott F. Walter wrote:> Jack Christensen wrote: > > >> If I''m going to run a task that may take a long time, how do I >> start a task and return without waiting for the task to finish? >> >> Thanks. >> >> > From my limited experience so far I don''t think Rails has this type > of functionality. Lets assume I am correct, so how would I > approach this? I would have a seperate ruby application (outside > of your Rails app) that would listen for soap requests. I would > probably start with have 2 different requests, a submit task > request and get task status request (returns percent complete). > Basically in a nutshell this other ruby application would be a > "task processing server". > You could then use ajax to poll for the status of a task to > determine if it has been completed or not. I have done something > like this in Java and it worked out well. In my case I built a > photo album generator. When the user would request to "build" the > photo album I would submit a request to the "taskserver". Every > few second I would ask the "taskserver" for the percent complete > and then display the percent complete on a web page. > > Anybody else have an idea? If not I would be to work with you on > an implementation strategy. > > -- > > Scott F. Walter Scott F. Walter > Principal Consultant > Vivare, Inc. > > E: scott.walter-uosFFu51klvQT0dZR+AlfA@public.gmane.org > E: scott-APWf0AbNa2kIjDr1QQGPvw@public.gmane.org > Visit scottwalter.com <http://scottwalter.com> --Point. Click. > Explore! > > > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >
I may be letting my anti-thread biases show here, but I would probably stay away from using a thread for this. Creating new threads will likely just end up increasing the complexity and fragility of the system (and I''m not even sure how thread-safe Rails code is). Also, the new thread will probably just slow down the next requests. As for collecting results from a singleton, that will only work if your next request is routed to the same process, which is not likely for FastCGI. I''m not sure what you are trying to accomplish, but I can think of a few options: (1) Fork a brand new process to handle the long running task. Communicate status information back to the user through database tables. (2) Have the web app write an entry in a table when a new task is requested. Have a cron that periodically polls the database for new tasks to run, updating the status in the database as in progress or completed. Rails works great from the command line. I usually opt for the second approach. This lets me run CPU intensive tasks on a dedicated build machine leaving the web servers free to respond to serve pages as quickly as possible. -Bob. On Aug 24, 2005, at 4:05 PM, Scott F. Walter wrote:> Ben Myles wrote: > > >> On 8/25/05, Scott F. Walter <scott.walter-uosFFu51klvQT0dZR+AlfA@public.gmane.org> wrote: >> >> >>> From my limited experience so far I don''t think Rails has this >>> type of >>> functionality. Lets assume I am correct, so how would I approach >>> this? >>> >>> >> >> um... >> >> def thread >> thread = Thread.new do >> while true >> rand(1000)*rand(1000)*rand(1000) >> end >> end >> render_text "thread going in background" >> end >> >> that what you''re after? (just put the code to execute in the >> thread block) >> >> Ben >> _______________________________________________ >> Rails mailing list >> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org >> http://lists.rubyonrails.org/mailman/listinfo/rails >> >> >> > Alot simpler than I first thought. So how do you let the user know > the process is done? I guess you could have some sort of Singleton- > design pattern based class that you could use to hold the status of > the long running process. That way you have something to poll to > let the user know the process is still going on and didn''t abruptly > fail. > > scott. > > -- > > Scott F. Walter Scott F. Walter > Principal Consultant > Vivare, Inc. > > E: scott.walter-uosFFu51klvQT0dZR+AlfA@public.gmane.org > E: scott-APWf0AbNa2kIjDr1QQGPvw@public.gmane.org > Visit scottwalter.com <http://scottwalter.com> --Point. Click. > Explore! > > > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >
Good points. Another "anti-thread" point is that if your FastCGI process dies so will your thread - so you can''t be guaranteed of its execution. I guess the best solution boils down to how long "long running" is. If you mean 10 seconds then I''d go for a thread, but if you are talking hours then definitely write a seperate application. Ben On 8/25/05, Robert Cottrell <bob-9FctsW8Z9av3oGB3hsPCZA@public.gmane.org> wrote:> I may be letting my anti-thread biases show here, but I would > probably stay away from using a thread for this. Creating new > threads will likely just end up increasing the complexity and > fragility of the system (and I''m not even sure how thread-safe Rails > code is). Also, the new thread will probably just slow down the next > requests. As for collecting results from a singleton, that will only > work if your next request is routed to the same process, which is not > likely for FastCGI. > > I''m not sure what you are trying to accomplish, but I can think of a > few options: > > (1) Fork a brand new process to handle the long running task. > Communicate status information back to the user through database tables. > > (2) Have the web app write an entry in a table when a new task is > requested. Have a cron that periodically polls the database for new > tasks to run, updating the status in the database as in progress or > completed. Rails works great from the command line. > > I usually opt for the second approach. This lets me run CPU > intensive tasks on a dedicated build machine leaving the web servers > free to respond to serve pages as quickly as possible. > > -Bob. > > > On Aug 24, 2005, at 4:05 PM, Scott F. Walter wrote: > > > Ben Myles wrote: > > > > > >> On 8/25/05, Scott F. Walter <scott.walter-uosFFu51klvQT0dZR+AlfA@public.gmane.org> wrote: > >> > >> > >>> From my limited experience so far I don''t think Rails has this > >>> type of > >>> functionality. Lets assume I am correct, so how would I approach > >>> this? > >>> > >>> > >> > >> um... > >> > >> def thread > >> thread = Thread.new do > >> while true > >> rand(1000)*rand(1000)*rand(1000) > >> end > >> end > >> render_text "thread going in background" > >> end > >> > >> that what you''re after? (just put the code to execute in the > >> thread block) > >> > >> Ben > >> _______________________________________________ > >> Rails mailing list > >> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > >> http://lists.rubyonrails.org/mailman/listinfo/rails > >> > >> > >> > > Alot simpler than I first thought. So how do you let the user know > > the process is done? I guess you could have some sort of Singleton- > > design pattern based class that you could use to hold the status of > > the long running process. That way you have something to poll to > > let the user know the process is still going on and didn''t abruptly > > fail. > > > > scott. > > > > -- > > > > Scott F. Walter Scott F. Walter > > Principal Consultant > > Vivare, Inc. > > > > E: scott.walter-uosFFu51klvQT0dZR+AlfA@public.gmane.org > > E: scott-APWf0AbNa2kIjDr1QQGPvw@public.gmane.org > > Visit scottwalter.com <http://scottwalter.com> --Point. Click. > > Explore! > > > > > > > > _______________________________________________ > > Rails mailing list > > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > > http://lists.rubyonrails.org/mailman/listinfo/rails > > > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >
On 25/08/2005, at 9:57 AM, Ben Myles wrote:> Good points. Another "anti-thread" point is that if your FastCGI > process dies so will your thread - so you can''t be guaranteed of its > execution.this was causing me a lot of problems a few months ago, my process would just stop. no logs, no errors, nothing. the dispatch.fcgi is better now, so i''d imagine it would be less painful to figure out whats going on...> > I guess the best solution boils down to how long "long running" is. If > you mean 10 seconds then I''d go for a thread, but if you are talking > hours then definitely write a seperate application.i went down the separate application path, and decided to use distributed-ruby (link eludes me right now). It gave me the benefits of a separate application, and also allowed me to push the whole process off to another server with very few problems. if you''re just using ruby, i''d recommend it... bodhi ===================================================Bodhi Philpot Lead Programmer GPlusMedia Co., Ltd. TEL 03-5475-7773 FAX 03-5475-7774 GPlusMedia Websites: http://www.gaijinpot.com/ Japan''s No.1 Jobsite for Foreigners http://www.ecentral.jp/ Japan''s Jobsite for Bilinguals http://www.openspace.jp/ Tokyo''s bilingual Real Estate Portal http://www.globalhealth.jp/ Health Insurance for Foreigners in Japan ====================================================
http://wiki.rubyonrails.com/rails/show/HowToRunBackgroundJobsInRails http://redhanded.hobix.com/inspect/daemonize.html personally i would fork over thread. and using something like memcached would make it extremely easy to get the status of a background job without killing your database server or knocking your harddrive to write a super tiny file all the time. On 8/25/05, bodhi <bodhi-DrnEsRrc7Su2oZ/6fjIToQ@public.gmane.org> wrote:> On 25/08/2005, at 9:57 AM, Ben Myles wrote: > > > Good points. Another "anti-thread" point is that if your FastCGI > > process dies so will your thread - so you can''t be guaranteed of its > > execution. > > this was causing me a lot of problems a few months ago, my process > would just stop. no logs, no errors, nothing. the dispatch.fcgi is > better now, so i''d imagine it would be less painful to figure out whats > going on... > > > > > I guess the best solution boils down to how long "long running" is. If > > you mean 10 seconds then I''d go for a thread, but if you are talking > > hours then definitely write a seperate application. > > i went down the separate application path, and decided to use > distributed-ruby (link eludes me right now). It gave me the benefits of > a separate application, and also allowed me to push the whole process > off to another server with very few problems. if you''re just using > ruby, i''d recommend it... > > bodhi > > ===================================================> Bodhi Philpot > Lead Programmer > GPlusMedia Co., Ltd. > TEL 03-5475-7773 > FAX 03-5475-7774 > GPlusMedia Websites: > http://www.gaijinpot.com/ Japan''s No.1 Jobsite for Foreigners > http://www.ecentral.jp/ Japan''s Jobsite for Bilinguals > http://www.openspace.jp/ Tokyo''s bilingual Real Estate Portal > http://www.globalhealth.jp/ Health Insurance for Foreigners in Japan > ===================================================> > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-- Zachery Hostens <zacheryph-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
I was just looking into this question yesterday, and the Pickaxe has a page on Rinda. Rinda is a distributed blackboard system like Linda, and it looks as though you can use this as an alternative to threads without load-related process forking. It has the added benefit of being cluster-transparent, so you could have one or one hundred compute nodes watching a blackboard. For my long-running (CPU-intensive) tasks, I''ll be using Rinda. -- Justin Dossey Operations, podOmatic.com On Thu, 25 Aug 2005, Julian Leviston wrote:> How about using Ajax to initiate it, and then monitoring a disk-file or > database record that the long-running task updates with a periodic-update > ajax call? > > Julian. > > On 25/08/2005, at 8:35 AM, Scott F. Walter wrote: > >> Jack Christensen wrote: >> >> >> > If I''m going to run a task that may take a long time, how do I start a >> > task and return without waiting for the task to finish? >> > >> > Thanks. >> > >> > >> From my limited experience so far I don''t think Rails has this type of > functionality. Lets assume I am correct, so how would I approach this? I > would have a seperate ruby application (outside of your Rails app) that would > listen for soap requests. I would probably start with have 2 different > requests, a submit task request and get task status request (returns percent > complete). Basically in a nutshell this other ruby application would be a > "task processing server". >> You could then use ajax to poll for the status of a task to determine if it >> has been completed or not. I have done something like this in Java and it >> worked out well. In my case I built a photo album generator. When the >> user would request to "build" the photo album I would submit a request to >> the "taskserver". Every few second I would ask the "taskserver" for the >> percent complete and then display the percent complete on a web page. >> >> Anybody else have an idea? If not I would be to work with you on an >> implementation strategy. >> >> -- >> >> Scott F. Walter Scott F. Walter >> Principal Consultant >> Vivare, Inc. >> >> E: scott.walter-uosFFu51klvQT0dZR+AlfA@public.gmane.org >> E: scott-APWf0AbNa2kIjDr1QQGPvw@public.gmane.org >> Visit scottwalter.com <http://scottwalter.com> --Point. Click. Explore! >> >> >> >> _______________________________________________ >> Rails mailing list >> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org >> http://lists.rubyonrails.org/mailman/listinfo/rails >> > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >