I''m using mechanize with separate agents in separate threads. I recently got the following error, and suspect that it may be a concurrency issue -- i.e., both threads were futzing with the same object. Anyone else think that could be the case? TypeError: can''t modify frozen object In /usr/local/lib/ruby/1.8/net/https.rb:138:in `verify_mode='' /usr/local/lib/ruby/1.8/net/https.rb:138:in `verify_mode='' /usr/local/lib/ruby/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/chain/ssl_resolver.rb:20:in `handle'' /usr/local/lib/ruby/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/chain.rb:30:in `pass'' /usr/local/lib/ruby/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/chain/handler.rb:6:in `handle'' /usr/local/lib/ruby/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/chain/connection_resolver.rb:73:in `handle'' /usr/local/lib/ruby/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/chain.rb:30:in `pass'' /usr/local/lib/ruby/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/chain/handler.rb:6:in `handle'' /usr/local/lib/ruby/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/chain/request_resolver.rb:27:in `handle'' /usr/local/lib/ruby/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/chain.rb:30:in `pass'' /usr/local/lib/ruby/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/chain/handler.rb:6:in `handle'' /usr/local/lib/ruby/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/chain/parameter_resolver.rb:18:in `handle'' /usr/local/lib/ruby/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/chain.rb:30:in `pass'' /usr/local/lib/ruby/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/chain/handler.rb:6:in `handle'' /usr/local/lib/ruby/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/chain/uri_resolver.rb:68:in `handle'' /usr/local/lib/ruby/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/chain.rb:25:in `handle'' /usr/local/lib/ruby/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize.rb:445:in `fetch_page'' /usr/local/lib/ruby/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize.rb:404:in `post_form'' /usr/local/lib/ruby/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize.rb:335:in `submit'' -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/mechanize-users/attachments/20100107/5107400a/attachment.html>
On Thu, Jan 7, 2010 at 1:32 PM, Aaron Starr <astarr at wiredquote.com> wrote:> > I''m using mechanize with separate agents in separate threads. I recently got > the following error, and suspect that it may be a concurrency issue -- i.e., > both threads were futzing with the same object. Anyone else think that could > be the case?Separate agents should be completely separate. I think this might be a bug specific to SSL requests. Would you mind grabbing the gem from github and trying it out? I think it may fix this problem. -- Aaron Patterson http://tenderlovemaking.com/
Using someone else''s experience and the results of my own futzing,
I''ve
ended up with the following monkey patch:
http://pastie.org/787553
Basically, in addition to checking for http_obj.frozen? it also checks for
http_obj.ssl_context.frozen?
It''s difficult for me to tell if it''s working, because
it''s only a rare,
intermittent problem. So far, so good. If someone is familiar with the code,
can you tell me if this is a really bad idea for some reason? In particular,
it''s not clear to me what should happen if the ssl_context is frozen.
Or,
why the http_obj might not be frozen when the ssl_context is. But
that''s
apparently what''s happening.
Aaron
On Thu, Jan 7, 2010 at 4:04 PM, Aaron Patterson
<aaron.patterson at gmail.com>wrote:
> On Thu, Jan 7, 2010 at 1:32 PM, Aaron Starr <astarr at
wiredquote.com> wrote:
> >
> > I''m using mechanize with separate agents in separate threads.
I recently
> got
> > the following error, and suspect that it may be a concurrency issue --
> i.e.,
> > both threads were futzing with the same object. Anyone else think that
> could
> > be the case?
>
> Separate agents should be completely separate. I think this might be
> a bug specific to SSL requests. Would you mind grabbing the gem from
> github and trying it out? I think it may fix this problem.
>
> --
> Aaron Patterson
> http://tenderlovemaking.com/
> _______________________________________________
> Mechanize-users mailing list
> Mechanize-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mechanize-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/mechanize-users/attachments/20100120/187b26a3/attachment-0001.html>
I have had a similar situation in a routine that we have been working on for the last few weeks. Our conclusion was the same as yours although we implemented it slightly differently. Obviously, we didn''t like monkey patching the system so we dug around a bit, and found that the issue has already been solved upstream in the following commits: http://github.com/tenderlove/mechanize/commit/f2fc38873c81f95472111da30b56d99c54946563 http://github.com/tenderlove/mechanize/commit/6ead88f3547fe012992460e455858ccade88d878 We did tests before and after those commits and the problem hasn''t occurred since. (We were getting "can''t modify frozen object" errors on nearly every run of our script.) I believe that Mechanize 0.9.3 was released around 2009/06/08 (per Gemcutter) . There has been a flurry of commits and merges in the last day or so. Any chance of an updated gem being pushed to gemcutter with all of the fixes, and new goodies? Robert Jackson On Jan 20, 2010, at 8:31 PM, Aaron Starr wrote:> > Using someone else''s experience and the results of my own futzing, > I''ve ended up with the following monkey patch: > > http://pastie.org/787553 > > Basically, in addition to checking for http_obj.frozen? it also > checks for http_obj.ssl_context.frozen? > > It''s difficult for me to tell if it''s working, because it''s only a > rare, intermittent problem. So far, so good. If someone is familiar > with the code, can you tell me if this is a really bad idea for some > reason? In particular, it''s not clear to me what should happen if > the ssl_context is frozen. Or, why the http_obj might not be frozen > when the ssl_context is. But that''s apparently what''s happening. > > Aaron > > > On Thu, Jan 7, 2010 at 4:04 PM, Aaron Patterson <aaron.patterson at gmail.com > > wrote: > On Thu, Jan 7, 2010 at 1:32 PM, Aaron Starr <astarr at wiredquote.com> > wrote: > > > > I''m using mechanize with separate agents in separate threads. I > recently got > > the following error, and suspect that it may be a concurrency > issue -- i.e., > > both threads were futzing with the same object. Anyone else think > that could > > be the case? > > Separate agents should be completely separate. I think this might be > a bug specific to SSL requests. Would you mind grabbing the gem from > github and trying it out? I think it may fix this problem. > > -- > Aaron Patterson > http://tenderlovemaking.com/ > _______________________________________________ > Mechanize-users mailing list > Mechanize-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mechanize-users > > _______________________________________________ > Mechanize-users mailing list > Mechanize-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mechanize-users-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/mechanize-users/attachments/20100121/87742feb/attachment.html>
On Thu, Jan 21, 2010 at 4:19 AM, Robert Jackson <robertj at promedicalinc.com> wrote:> I have had a similar situation in a routine that we have been working on for > the last few weeks. ?Our conclusion was the same as yours although we > implemented it slightly differently. > Obviously, we didn''t like monkey patching the system so we dug around a bit, > and found that the issue has already been solved upstream in the following > commits: > http://github.com/tenderlove/mechanize/commit/f2fc38873c81f95472111da30b56d99c54946563 > http://github.com/tenderlove/mechanize/commit/6ead88f3547fe012992460e455858ccade88d878 > We did tests before and after those commits and the problem hasn''t occurred > since. (We were getting "can''t modify frozen object"?errors on nearly every > run of our script.) > I believe that Mechanize 0.9.3 was released around 2009/06/08 (per > Gemcutter) . ?There has been a flurry of commits and merges in the last day > or so. > Any chance of an updated gem being pushed to gemcutter with all of the > fixes, and new goodies?Ya, I''m gearing up for a 1.0 release soon, so if folks want stuff in the release, send me your patches. :-) I''m not sure on the exact date, but sometime before Feb 18th. -- Aaron Patterson http://tenderlovemaking.com/
Oh, thanks for the heads-up. I looked at the code on github, but I''m not especially clever, and for some reason I didn''t see these updated versions of the file. I appreciate the response. Aaron On Thu, Jan 21, 2010 at 4:19 AM, Robert Jackson <robertj at promedicalinc.com>wrote:> I have had a similar situation in a routine that we have been working on > for the last few weeks. Our conclusion was the same as yours although we > implemented it slightly differently. > > Obviously, we didn''t like monkey patching the system so we dug around a > bit, and found that the issue has already been solved upstream in the > following commits: > > > http://github.com/tenderlove/mechanize/commit/f2fc38873c81f95472111da30b56d99c54946563 > > http://github.com/tenderlove/mechanize/commit/6ead88f3547fe012992460e455858ccade88d878 > > We did tests before and after those commits and the problem hasn''t occurred > since. (We were getting "can''t modify frozen object" errors on nearly every > run of our script.) > > I believe that Mechanize 0.9.3 was released around 2009/06/08 (per > Gemcutter) . There has been a flurry of commits and merges in the last day > or so. > > Any chance of an updated gem being pushed to gemcutter with all of the > fixes, and new goodies? > > Robert Jackson > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/mechanize-users/attachments/20100121/5ba673a7/attachment.html>