[Mechanize-users] Does Amazon.com blocks scraping?

Felipe Jordão A. P. Mattosinho
2010-01-25T05:29:33Z

Hi there



Does anyone know if Amazon.com has any sort of server side script that tries
to block scraping activities?  I first noticed that if I didn?t change the
agent alias, it would fetch a page exactly like the normal one, but without
the intial search field(maybe a silly way to  prevent scraping). Then after
it, I changed to some other alias, and submit a search. I got the result
page as response, but right after getting the page, I received a message
that Amazon.com closed my connection, and redirects me to another place.

If anyone succeeded with Amazon.com, circunventing this protection, please
send me some info



Regards,



Felipe



-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/mechanize-users/attachments/20100125/8535fc9d/attachment.html>