Mechanize users - Sep 2012 - Mechanize and Thread-Safety Revisited

This post from Aaron Patterson back in 2009 seems to be pretty definite about
how to use Mechanize in a thread-safe way, but I thought I would revisit the
question because there are several posts on StackOverflow since then that would
seem to contradict it (of course, I trust Aaron''s answer more than
random answers on StackOverlfow, but just in case I thought I would confirm it).

It is still the case that if you do not share agent instances across threads
then Mechanize IS thread-safe? Are there any other caveats to be aware of?


-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/mechanize-users/attachments/20120922/eddea3a5/attachment.html>

On Sep 22, 2012, at 11:56 AM, Jack Royal-Gordon <jack at
groundbreakingsoftware.com> wrote:
> This post from Aaron Patterson back in 2009 seems to be pretty definite
about how to use Mechanize in a thread-safe way
Which post?  You must include the URL.
> I thought I would revisit the question because there are several posts on
StackOverflow since then that would seem to contradict it (of course, I trust
Aaron''s answer more than random answers on StackOverlfow, but just in
case I thought I would confirm it).
Mechanize is not thread-safe.  At least the history and the cookie jar features
would require mutual exclusion for safe behavior.
> It is still the case that if you do not share agent instances across
threads then Mechanize IS thread-safe? Are there any other caveats to be aware
of?
Separate Mechanize instances don''t share state with each other, so one
instance per thread is fine.  By default mechanize caches the most recent 50
pages visited per instance.  For large responses this consumes file descriptors,
for small responses this consumes memory.  If you''re using many threads
you will want to reduce the size of history or adjust the response buffering
appropriately.

FYI, this is the post the OP linked to:

http://rubyforge.org/pipermail/mechanize-users/2009-September/000449.html

Thanks for the tip about history. FWIW, we''ve been using one agent per
thread for a few years now, and it seems to be working well. (As both you,
Eric, and Aaron Patterson said it would.)

Best,

Aaron


On Mon, Sep 24, 2012 at 10:53 AM, Eric Hodel <drbrain at segment7.net>
wrote:
> On Sep 22, 2012, at 11:56 AM, Jack Royal-Gordon <
> jack at groundbreakingsoftware.com> wrote:
> > This post from Aaron Patterson back in 2009 seems to be pretty
definite
> about how to use Mechanize in a thread-safe way
>
> Which post?  You must include the URL.
>
> > I thought I would revisit the question because there are several posts
> on StackOverflow since then that would seem to contradict it (of course, I
> trust Aaron''s answer more than random answers on StackOverlfow,
but just in
> case I thought I would confirm it).
>
> Mechanize is not thread-safe.  At least the history and the cookie jar
> features would require mutual exclusion for safe behavior.
>
> > It is still the case that if you do not share agent instances across
> threads then Mechanize IS thread-safe? Are there any other caveats to be
> aware of?
>
> Separate Mechanize instances don''t share state with each other, so
one
> instance per thread is fine.  By default mechanize caches the most recent
> 50 pages visited per instance.  For large responses this consumes file
> descriptors, for small responses this consumes memory.  If you''re
using
> many threads you will want to reduce the size of history or adjust the
> response buffering appropriately.
> _______________________________________________
> Mechanize-users mailing list
> Mechanize-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mechanize-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/mechanize-users/attachments/20120924/adb48d98/attachment-0001.html>