Tom Lester
2007-Feb-26 22:46 UTC
[Mechanize-users] some Mechanize objects never garbage collected?
Greetings, I''m using Mechanize to scrap dozens of pages and have noticed the size of my ruby process keeps growing. I set Mechanize.max_history to 0 with no effect on the memory use. I wrote a little test to show the objects left on the heap after mechanizing a single page and then doing a garbage collection. Sample list appended below. I can supply the test code if it helps. I did a little poking around the code but thought I would ask here before investigating more. Is this a know problem? Cheers, Tom Objects left on heap after mechanizing a single page and doing a garbage collection: Hpricot::Text 18 | delta 18 Hpricot::STag 12 | delta 12 Hpricot::Elem 12 | delta 12 Hpricot::ETag 8 | delta 8 WWW::Mechanize::List 5 | delta 5 WWW::Mechanize::Link 2 | delta 2 MatchData 2 | delta 1 Thread 2 | delta 1 WWW::Mechanize::Page 2 | delta 2 Net::HTTPOK 1 | delta 1 URI::HTTP 1 | delta 1 Net::HTTP 1 | delta 1 LoadError 1 | delta -1 Hpricot::Doc 1 | delta 1 WWW::Mechanize::PluggableParser 1 | delta 1 StringIO 1 | delta 1 Net::ReadAdapter 1 | delta 1 Net::HTTP::Get 1 | delta 1 WWW::Mechanize 1 | delta 1 WWW::Mechanize::CookieJar 1 | delta 1
Aaron Patterson
2007-Feb-27 00:18 UTC
[Mechanize-users] some Mechanize objects never garbage collected?
Hi Tom, On Mon, Feb 26, 2007 at 02:46:28PM -0800, Tom Lester wrote:> Greetings, > > I''m using Mechanize to scrap dozens of pages and have noticed the > size of my ruby process keeps growing. I set Mechanize.max_history to > 0 with no effect on the memory use. I wrote a little test to show the > objects left on the heap after mechanizing a single page and then > doing a garbage collection. Sample list appended below. I can supply > the test code if it helps. > > I did a little poking around the code but thought I would ask here > before investigating more. Is this a know problem?Yes, this is a known problem and will be fixed in version 0.6.5. I will have that released soon since there are no more open bugs. If you need it fixed right away, you can use 0.6.3 which does not have this bug. -- Aaron Patterson http://tenderlovemaking.com/