Gavin Morrice wrote:> Hey all
>
> I''m experimenting with writing a scraper at the moment and have
hit a
> major hump.
>
> Part of the DOM is added after the page has loaded via javascript.
>
> This means when I make an a request the HTML response I receive back
> doesn''t accurately represent the page.
>
> Here''s a simplified example:
>
> @http_obj = Net::HTTP.new("targetdomain.com")
>
> response, page_data = @http_obj.request_get( "/" )
>
> # page data doesn''t contain all of the HTML that is actually shown
>
> Is there anyway library or gem that could simulate the browser
> updating the DOM with the Javascript or any other way I could approach
> this short of decoding the obfuscated Javascript file?
Try Selenium or some other remote browser control.
>
> Thanks in advance
>
> Gav
Best,
--
Marnen Laibow-Koser
http://www.marnen.org
marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org
--
Posted via http://www.ruby-forum.com/.
--
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.