Greetings, I''m reporting on what I found after trying to use mechanize on a site like www.tellme.com. With mechanize versions 0.6.5, 0.6.4, 0.6.3, I was able to use mechanize without any problems on www.tellme.com. However, when I upgraded to 0.6.6 or 0.6.7, mechanize simply ground to a halt after a while. I''m not sure where the problem lies, but after looking at the changelogs for 0.6.6 and 0.6.7, it seems that keep-alives might be the culprit here. In any case, I''m back to using 0.6.5 and the app is working fine. Hopefully this helps somebody who is using mechanize. cheers, mengkuan
On Fri, Mar 30, 2007 at 10:30:16AM +0800, Meng Kuan wrote:> Greetings, > > I''m reporting on what I found after trying to use mechanize on a site > like www.tellme.com. > > With mechanize versions 0.6.5, 0.6.4, 0.6.3, I was able to use > mechanize without any problems on www.tellme.com. > > However, when I upgraded to 0.6.6 or 0.6.7, mechanize simply ground > to a halt after a while. I''m not sure where the problem lies, but > after looking at the changelogs for 0.6.6 and 0.6.7, it seems that > keep-alives might be the culprit here. > > In any case, I''m back to using 0.6.5 and the app is working fine.Interesting. Can you tell if it was the script, or the server that stopped working? Were there any errors? What was the memory usage like? Also, is it possible for you to write a short sample reproducing the error? Thanks! -- Aaron Patterson http://tenderlovemaking.com/
Hi Aaron, On 30 Mar 2007, at 12:58 PM, Aaron Patterson wrote:> > Interesting. Can you tell if it was the script, or the server that > stopped working? Were there any errors? What was the memory usage > like? >No errors. I was actually running the mechanize process off of backgroundrb. Memory utilization was reasonable. Running "top" shows the process using up around 1.8% of memory (works out to about 22 MB on my system). However, it does not exit properly so backgroundrb thinks it is still running and the worker process hangs around indefinitely so I had to kill it manually.> Also, is it possible for you to write a short sample reproducing the > error?Attached spider.rb file contains the test Spider class I use for this testing. Use it like this: s = Spider.new(''http://www.tellme.com'') s.process The mech_test.log file shows where the process abruptly exited for version 0.6.7. I tried the same script on another system with 0.6.4 and it ran to completion. I''m wondering if the state of the tellme.com server is to blame for this. cheers, mengkuan -------------- next part -------------- A non-text attachment was scrubbed... Name: spider.rb Type: text/x-ruby-script Size: 2663 bytes Desc: not available Url : http://rubyforge.org/pipermail/mechanize-users/attachments/20070330/1a57b167/attachment-0001.bin -------------- next part -------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: mech_test.log Type: application/octet-stream Size: 85166 bytes Desc: not available Url : http://rubyforge.org/pipermail/mechanize-users/attachments/20070330/1a57b167/attachment-0001.obj -------------- next part --------------