Rails - Oct 2006 - Design Dilemma - Please Help

Hi, I''m new. ;-)

I creating a little rails app, that will crawl the web on a regular
basis and then show the results.

The crawling will be scheduled, likely a cron job.

I can''t wrap my head around where to put my crawler.  It
doesn''t seem
to fit.

An example:
Model - News Story
Controllers - Grabs a story from the DB, Sort the Stories, Search the
Stories etc.
View - HTML News Story, RSS Story etc.

Then a I have a news crawler, that will go crawl some feeds for new
stories, then insert them into the db. Where do I put it, and how do I
get cron to execute it?

Maybe put it in the NewsStoryController?

Do I make another file for cron to run, that contains something like
this:
---
nc = NewsStoryController.new
nc.crawl
---

If I do this will I still have access to the ActiveRecord the same as
usual?

I''m confused, and any guidance or feedback would be awesome!

Thanks,

Chris


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

On Oct 23, 2006, at 12:06 AM, Chris.Mohr wrote:
> Then a I have a news crawler, that will go crawl some feeds for new
> stories, then insert them into the db. Where do I put it, and how do I
> get cron to execute it?
>
> Maybe put it in the NewsStoryController?
>
> Do I make another file for cron to run, that contains something like
> this:
> ---
> nc = NewsStoryController.new
> nc.crawl
> ---
>
> If I do this will I still have access to the ActiveRecord the same as
> usual?
>
> I''m confused, and any guidance or feedback would be awesome!
I''d place it in the class of it''s own, in a file under lib/
and
use script/runner to invoke it.

Crawl would be a class method, so you would invoke it like this:

   script/runner ''NewsStoryController.crawl''

or, in production

   script/runner -e production ''NewsStoryController.crawl''

-- 
-- Tom Mornini


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

Awesome...thank you.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

On Oct 23, 2006, at 2:06 AM, Chris.Mohr wrote:
> I creating a little rails app, that will crawl the web on a regular  
> basis and then show the results.
>
> The crawling will be scheduled, likely a cron job.
>
> I can''t wrap my head around where to put my crawler.  It
doesn''t
> seem to fit.
An alternative to the suggestion already offered would be to use  
backgroundrb [1] and to set up your crawler as a worker class in

lib/workers

I''m using that to good effect in an application (not yet launched)  
which makes heavy use of regular screen scraping.

James.

1: http://backgroundrb.rubyforge.org/
-- 
James Stewart : Web Developer
Work : http://jystewart.net
Play : http://james.anthropiccollective.org




--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---