Mechanize users - Mar 2010 - tutorial for monitoring browser <=> server exchanges?

I''ve dabbled in Mechanize and love it.  My application requires page  
scrubbing from different sites, each with its own method for logging  
in, redirecting, setting cookies, etc.

Are there any good tutorials (and perhaps tools) for watching the  
exchanges between a browser and a server as a user logs into a site  
and accesses a specific page?  Once I can see those exchanges, I''ll  
know what Mechanize needs to do.

Thanks in advance...

- ff

Wireshark is my favorite.



On Thu, Mar 25, 2010 at 10:06 AM, Robert Poor <rdpoor at gmail.com>
wrote:
> I''ve dabbled in Mechanize and love it. ?My application requires
page
> scrubbing from different sites, each with its own method for logging in,
> redirecting, setting cookies, etc.
>
> Are there any good tutorials (and perhaps tools) for watching the exchanges
> between a browser and a server as a user logs into a site and accesses a
> specific page? ?Once I can see those exchanges, I''ll know what
Mechanize
> needs to do.
>
> Thanks in advance...
>
> - ff
> _______________________________________________
> Mechanize-users mailing list
> Mechanize-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mechanize-users
>

On Thu, Mar 25, 2010 at 9:06 AM, Robert Poor <rdpoor at gmail.com>
wrote:
> I''ve dabbled in Mechanize and love it. ?My application requires
page
> scrubbing from different sites, each with its own method for logging in,
> redirecting, setting cookies, etc.
>
> Are there any good tutorials (and perhaps tools) for watching the exchanges
> between a browser and a server as a user logs into a site and accesses a
> specific page? ?Once I can see those exchanges, I''ll know what
Mechanize
> needs to do.
Typically, I use LiveHTTPHeaders or Firebug to watch the request and
responses.  But if you have an end user doing this, they might not be
so technically inclined.  I''ve been messing with a way to record
sessions:

  http://gist.github.com/309164

Basically, it sets up a proxy, you tell your browser to use that
proxy, and the proxy records your interactions.  You might be able to
modify that code to get it to record your user''s actions as a
mechanize script.

-- 
Aaron Patterson
http://tenderlovemaking.com/

I''ve used Charles (http://www.charlesproxy.com/) with good success. You
can
write your sessions out to an xml file that captures everything about the
session.

Depending on your need, you may find it useful to write a script to
translate the xml into mechanize code. Or, you can just read it yourself.

Aaron


On Thu, Mar 25, 2010 at 9:06 AM, Robert Poor <rdpoor at gmail.com> wrote:
> I''ve dabbled in Mechanize and love it.  My application requires
page
> scrubbing from different sites, each with its own method for logging in,
> redirecting, setting cookies, etc.
>
> Are there any good tutorials (and perhaps tools) for watching the exchanges
> between a browser and a server as a user logs into a site and accesses a
> specific page?  Once I can see those exchanges, I''ll know what
Mechanize
> needs to do.
>
> Thanks in advance...
>
> - ff
> _______________________________________________
> Mechanize-users mailing list
> Mechanize-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mechanize-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/mechanize-users/attachments/20100325/1abb956f/attachment.html>