Suppose, for a moment, that I have a Camping app. It looks at something, like an RSS feed of upcoming events, and displays them if they meet a specified criterion. I look at the index page, it fetches the feed items and then displays them. Yay. Maybe it also writes them to a database. Which is great, so long as I keep looking at the page every so often. But what do I do while I''m sleeping? or on holiday? Who will look at the page to fetch the feeds then? So, is there some way I can make Camping perform the fetch operation regularly, all internal to Camping? I know I could set up a cron job to grab the page via curl every hour or whatever, but can I do it just in Camping? Any pointers gratefully recieved, Jonathan
> Which is great, so long as I keep looking at the page every so often. > But what do I do while I''m sleeping? or on holiday? Who will look at > the page to fetch the feeds then?If a feed gets updated in the woods, but there''s nobody there to read it, is there any new content? ;-) But I understand what you''re saying: you want to fetch the feeds so you don''t miss any items that fall off the bottom of the feed and/or the performance hit of fetching the feed each time is irritating.> > So, is there some way I can make Camping perform the fetch operation > regularly, all internal to Camping? I know I could set up a cron job > to grab the page via curl every hour or whatever, but can I do it just > in Camping?Two proposed solutions: 1. Threads: create a thread that just sleeps for an hour, then wakes up and fetches the feed. You can do this in your camping app. I''m not a threading expert, so I''ll leave it to you to fill in the details. 2. External cron job: Every hour run something like `curl http://myapp.com/fetchfeeds > /dev/null` 2 is probably the quick and dirty fix. 1 is certainly more clever. Good luck -Mark
On Apr 25, 2007, at 3:57, Mark Fredrickson wrote:> > 2. External cron job: Every hour run something like `curl > http://myapp.com/fetchfeeds > /dev/null`I would recommend a cronjob, solving it with a long running process will likely get you in trouble with memory leaks and such. Manfred
> I would recommend a cronjob, solving it with a long running process > will likely get you in trouble with memory leaks and such. > > ManfredI''m not quite sure how this is relevant (though I fully admit I might be missing some of the complexities of ruby, camping and/or threads), since I''d like the process to be part of a Camping App, which is presumably going to be running fairly consistently anyway. Unless you''re suggesting an entirely external app that does the fetching and somehow pushes the results into the database of the camping application so it can display it? Anyway, I know I can set up a cronjob with curl to regularly get the page and thus make the controller logic fetch the feeds, I was just wondering if there was a way to do it in the app itself, since I like applications that are self-contained. Jonathan
On 4/25/07, Jonathan Stott <jonathan.stott at gmail.com> wrote:> > I would recommend a cronjob, solving it with a long running process > > will likely get you in trouble with memory leaks and such. > > > > Manfred > > I''m not quite sure how this is relevant (though I fully admit I might > be missing some of the complexities of ruby, camping and/or threads), > since I''d like the process to be part of a Camping App, which is > presumably going to be running fairly consistently anyway. Unless > you''re suggesting an entirely external app that does the fetching and > somehow pushes the results into the database of the camping > application so it can display it?The problem with long-running multi-threaded Ruby processes tends to be that the threads themselves fail to be garbage collected, eventually resulting in memory problems. If you only have one background thread doing the updates, you really shouldn''t have to worry about it as a major source of leaks.> Anyway, I know I can set up a cronjob with curl to regularly get the > page and thus make the controller logic fetch the feeds, I was just > wondering if there was a way to do it in the app itself, since I like > applications that are self-contained.It really shouldn''t be hard. Something like the following in your application startup code should be enough: --- # Note: this line needs to come before Camping does its auto-magical SQLite # connection stuff, so you may need to experiment ActiveRecord::Base.allow_concurrency = true # This line just starts a background thread that will run once per hour in the # background, calling your refresh method each time Thread.new { loop { MyApp::Controllers::FeedReader.refresh; sleep 3600 } } --- -Lennon
2007/4/25, Jonathan Stott <jonathan.stott at gmail.com>:> Suppose, for a moment, that I have a Camping app. It looks at > something, like an RSS feed of upcoming events, and displays them if > they meet a specified criterion. I look at the index page, it fetches > the feed items and then displays them. Yay. Maybe it also writes them > to a database. > > Which is great, so long as I keep looking at the page every so often. > But what do I do while I''m sleeping? or on holiday? Who will look at > the page to fetch the feeds then? > > So, is there some way I can make Camping perform the fetch operation > regularly, all internal to Camping? I know I could set up a cron job > to grab the page via curl every hour or whatever, but can I do it just > in Camping?You can make use of the create method to initialize your thread. Just make sure that you store the thread''s presence outside of the app''s module to avoid problems when reloading.. module YourApp def self.create ::YOURAPP_FETCH_THREAD ||= Thread.new { ... } end end PS : I wouldn''t use an external scheduler if the database is sqlite.. I had lots of problems with locking. -- Cheers, zimbatm
> You can make use of the create method to initialize your thread. Just > make sure that you store the thread''s presence outside of the app''s > module to avoid problems when reloading.. > > module YourApp > def self.create > ::YOURAPP_FETCH_THREAD ||= Thread.new { ... } > end > end > > PS : I wouldn''t use an external scheduler if the database is sqlite.. > I had lots of problems with locking. > > -- > Cheers, > zimbatmThank you Lennon and Jonas, everything seems to be working as I wanted it to now :) Regards Jonathan