Hi guys, I use Nokogiri to fetch some HTML pages and scrape it in the background. So far it works fine. But now I have to login to view some other pages in the background. How can I login in the background to fetch some HTML? In other words how can I login in the background using Rails? Note that I''m not talking about some login form or something. -- Posted via http://www.ruby-forum.com/. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Hi! I don''t know if I understood you well, but take a lookt at mechanize: http://railscasts.com/episodes/191-mechanize Regards, Everaldo On Mon, Nov 14, 2011 at 2:01 PM, Jeroen van Ingen <lists-fsXkhYbjdPsEEoCn2XhGlw@public.gmane.org>wrote:> Hi guys, > > I use Nokogiri to fetch some HTML pages and scrape it in the background. > So far it works fine. But now I have to login to view some other pages > in the background. How can I login in the background to fetch some HTML? > In other words how can I login in the background using Rails? > > Note that I''m not talking about some login form or something. > > -- > Posted via http://www.ruby-forum.com/. > > -- > You received this message because you are subscribed to the Google Groups > "Ruby on Rails: Talk" group. > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To unsubscribe from this group, send email to > rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > For more options, visit this group at > http://groups.google.com/group/rubyonrails-talk?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Mechanize looks good, I''ll definitely give it a try. Where I''m basically searching for is a gem or library that can store a session / cookie, so Rails can login on a website in the background. Anyone who know if there are other options besides mechanize? -- Posted via http://www.ruby-forum.com/. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Jeroen van Ingen wrote in post #1031839:> Mechanize looks good, I''ll definitely give it a try. > > Where I''m basically searching for is a gem or library that can store a > session / cookie, so Rails can login on a website in the background. > > Anyone who know if there are other options besides mechanize?Hi Jeroen: It seems you''re talking about two separate things: (a) a mechanism to log in and scrape pages (b) a mechanism to run a task in the background For (a) I use Mechanize for logging in and page scraping -- that''s what it''s designed for. For (b), I use DelayedJob. I''ve had no problem running a Mechanize task in the background using DelayedJob. - ff -- Posted via http://www.ruby-forum.com/. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.