Mechanize users - Aug 2009 - Why won't this Ruby/Mechanize HTTP POST work? (the Bank's web server gives Session Timeout response)

Hi,

A question for the web/http guru''s out there if I may (who understand
cookies, sessions ids, etc).

BACKGROUND:
=========I''m writing some Ruby code (using Mechanize) to automate
logging into
my bank account to check the balance.  The first step after I
successfully log and click on a button am getting "You Session Has
Ended"!  I''ve simulated using the bank account on my browser
manually
without Javascript enabled so this shouldn''t be an issue (I note that
someone posted Mechanize doesn''t support javascript).  I''m
wondering
what it could possibly be & what tools/approach I could you to prove
the request from Ruby/Mechanize exactly matches the succesful post
that occurs when I log in through my browser.

QUESTIONS:
=========Q1 - Based on the below can someone see why I am getting a "You
Session Has Ended"?  (e.g. it it something to do with
SessionId/cookies/full post paremeters being in place).   Some ideas,
however not sure if they are significant:
(a) I had to do some work arounds as Mechanize didn''t seem to be
creating a valid URI re the first part of the URL, hence I hand
modified it.  I think it should be OK?  It the line in the code
"continue_form.action = "https://banking2.anz.com/IBAU/" +
continue_form.action"
(b) The form itself (see below) actually doesn''t have a SUBMIT button
but has a "INPUT TYPE="Image" tag.  Mechanize doesn''t
seem to pick
this up, but when I do mechanize "submit" via the code "page
agent.submit(continue_form)" it still seems to initiate a POST to the
server, albeit the response that comes back says the session has
expired.  I note there are some missing parameters in the browser
trace versus the Ruby/Mechanize trace (see below), but would this
cause a "session is no longer valid"?  (I tried to manually add fields
to compensate into the form via the code
"continue_form.add_field!("Action.RetUser.SignonOK","Proceed
to
Internet Banking")" however it didn''t seem to trigger them to
be
included)

Q2 - What tool can I use on both the Browser & my Ruby/Mechanize code
to compare the exact POSTs and see where the differences are?    I can
do Live HTTP Headers in firefox for the browser test (which works),
but in Ruby/Mechanize I think I''m stuck with it''s logging.  I
can''t
run WireShark as the traffic is HTTPS and encrypted. Any ideas?

Q3 - If the exact issues isn''t obvious any suggestions/ideas re what to
try?

FURTHER BACKGROUND:
=================* Successful Trace - manually via browser - using HTTP Live
Headers
plugin for Firefox (I''ve shorted session key for succintness)
POST /IBAU/BANKAWAYTRAN;jsessionid=xxx
Action.RetUser.SignonOK.x=39&Action.RetUser.SignonOK.y=12&Action.RetUser.SignonOK=Proceed+to+Internet+Banking

* Unsuccessful Trace - i.e. What I see in the Ruby/Mechanize log file
Net::HTTP::Post: /IBAU/BANKAWAYTRAN;jsessionid=xxx

* Guts of the Form I''m simulating the response for:
<FORM ACTION="BANKAWAYTRAN;jsessionid=xxx" METHOD="POST"
NAME="securityForm">
    <INPUT TYPE="Image"
    SRC="web/L001/images/00010/b_proceedIB_99CC00.gif"
    VALUE="Proceed to Internet Banking"
    alt="Proceed to Internet Banking"
    NAME="Action.RetUser.SignonOK">
    </input>
</FORM>

* My Ruby/Mechanize Code
--------------------------------------------------------------------------------
require ''rubygems''
require ''mechanize''
require ''mechanize_extns/form_extn.rb''
require ''logger''

agent = WWW::Mechanize.new{|a| a.log = Logger.new(STDERR) }

# Step 1 - Navigate to Login Page
page = agent.get(''https://www.anz.com/INETBANK/login.asp'')
if !page.forms[1] || (page.forms[1].name != "loginForm")
  puts "ERROR: At login form validation failed"
  exit
end

# Step 2 - Login
login_form = page.forms[1]
login_form.CorporateSignonCorpId = "xxxx"
login_form.CorporateSignonPassword = "yyyy"
page = agent.submit(login_form)
if !page.forms[0] || (page.forms[0].name != "securityForm")
  puts "ERROR: At clicking on continue button after login"
  exit
end

# Step 3 - Click Continue to Main Page  (** WHERE ISSUE OCCURS **)
continue_form = page.forms[0]
# work around 1 - the bank''s input tage of type image does not seem to
get handled, so add a button in
    continue_form.add_field!("Action.RetUser.SignonOK","Proceed
to
Internet Banking")  # Don''t see effect of this in log trace?
   
continue_form.add_field!("Action.RetUser.SignonOK.x","23")
                           # Don''t see effect of this in log trace?
   
continue_form.add_field!("Action.RetUser.SignonOK.y","7")
                            # Don''t see effect of this in log trace?
# work around 2 - the relative URI in the bank form''s action does not
seem to get handled, so add the front part on
    continue_form.action = "https://banking2.anz.com/IBAU/" +
continue_form.action
page = agent.submit(continue_form)
if !page.forms[1] || (page.forms[1].name != "formAll1")
  puts "ERROR: At clicking on continue button after login
=================================="    # <== FAILED HERE WITH SESSION
ENDED IN REPONSE
  puts page.content
  puts "ERROR: At clicking on continue button after login
=================================="
  exit
end


# Step Final - Logout
puts "Completed!"
--------------------------------------------------------------------------------



Any other info that would help you out helping me out?

thanks

You should look into Charles: http://www.charlesproxy.com/

This will allow you to watch the encrypted traffic from your browser, so you
can see exactly what cookies, parameters, etc., are being passed back and
forth. I''d be a little surprised if the login for a bank consisted of a
single POST transaction, since in my experience these things tend to consist
of multiple redirects and cookies set through javascript and other nonsense.

Aaron


[...]

Q2 - What tool can I use on both the Browser & my Ruby/Mechanize
code
> to compare the exact POSTs and see where the differences are?    I can
> do Live HTTP Headers in firefox for the browser test (which works),
> but in Ruby/Mechanize I think I''m stuck with it''s
logging.  I can''t
> run WireShark as the traffic is HTTPS and encrypted. Any ideas?
> [...]
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/mechanize-users/attachments/20090829/f711a6ce/attachment.html>

On Sat, Aug 29, 2009 at 3:53 AM, Greg
Hauptmann<greg.hauptmann.ruby at gmail.com> wrote:
> Hi,
>
> A question for the web/http guru''s out there if I may (who
understand
> cookies, sessions ids, etc).
>
> BACKGROUND:
> =========> I''m writing some Ruby code (using Mechanize) to
automate logging into
> my bank account to check the balance. ?The first step after I
> successfully log and click on a button am getting "You Session Has
> Ended"! ?I''ve simulated using the bank account on my browser
manually
> without Javascript enabled so this shouldn''t be an issue (I note
that
> someone posted Mechanize doesn''t support javascript).
?I''m wondering
> what it could possibly be & what tools/approach I could you to prove
> the request from Ruby/Mechanize exactly matches the succesful post
> that occurs when I log in through my browser.
I typically use LiveHTTPHeaders along with the mechanize logs.
> QUESTIONS:
> =========> Q1 - Based on the below can someone see why I am getting a
"You
> Session Has Ended"? ?(e.g. it it something to do with
> SessionId/cookies/full post paremeters being in place). ? Some ideas,
> however not sure if they are significant:
> (a) I had to do some work arounds as Mechanize didn''t seem to be
> creating a valid URI re the first part of the URL, hence I hand
> modified it. ?I think it should be OK? ?It the line in the code
> "continue_form.action = "https://banking2.anz.com/IBAU/" +
> continue_form.action"
Modifying the form action is fine as long as you *know* that is the
correct post URL.
> (b) The form itself (see below) actually doesn''t have a SUBMIT
button
> but has a "INPUT TYPE="Image" tag. ?Mechanize
doesn''t seem to pick
> this up, but when I do mechanize "submit" via the code "page
> agent.submit(continue_form)" it still seems to initiate a POST to the
> server, albeit the response that comes back says the session has
> expired. ?I note there are some missing parameters in the browser
> trace versus the Ruby/Mechanize trace (see below), but would this
> cause a "session is no longer valid"? ?(I tried to manually add
fields
> to compensate into the form via the code
>
"continue_form.add_field!("Action.RetUser.SignonOK","Proceed
to
> Internet Banking")" however it didn''t seem to trigger
them to be
> included)
Missing post parameters are probably the cause.  Treat the form like a
hash though:

  continue_form[''Action.RetUser.SignonOK''] =
''Whatever''

Make sure the the post parameters from LiveHTTPHeaders match the ones
output in the mechanize logs.
> Q2 - What tool can I use on both the Browser & my Ruby/Mechanize code
> to compare the exact POSTs and see where the differences are? ? ?I can
> do Live HTTP Headers in firefox for the browser test (which works),
> but in Ruby/Mechanize I think I''m stuck with it''s
logging. ?I can''t
> run WireShark as the traffic is HTTPS and encrypted. Any ideas?
Unless you run Mechanize through a proxy, it''s logs are the best tool.
 What are you missing from the logs?  The logs should give you all
http headers back and forth, along with any post data.  Seeing the
exact headers sent back and forth is paramount to making mechanize
match the browser.
> Q3 - If the exact issues isn''t obvious any suggestions/ideas re
what to try?
>
> FURTHER BACKGROUND:
> =================> * Successful Trace - manually via browser - using
HTTP Live Headers
> plugin for Firefox (I''ve shorted session key for succintness)
> POST /IBAU/BANKAWAYTRAN;jsessionid=xxx
>
Action.RetUser.SignonOK.x=39&Action.RetUser.SignonOK.y=12&Action.RetUser.SignonOK=Proceed+to+Internet+Banking
>
> * Unsuccessful Trace - i.e. What I see in the Ruby/Mechanize log file
> Net::HTTP::Post: /IBAU/BANKAWAYTRAN;jsessionid=xxx
You probably need those extra parameters.  Also make sure that the
cookies sent back and forth look correct.
> * Guts of the Form I''m simulating the response for:
> <FORM ACTION="BANKAWAYTRAN;jsessionid=xxx"
METHOD="POST" NAME="securityForm">
> ? ?<INPUT TYPE="Image"
> ? ?SRC="web/L001/images/00010/b_proceedIB_99CC00.gif"
> ? ?VALUE="Proceed to Internet Banking"
> ? ?alt="Proceed to Internet Banking"
> ? ?NAME="Action.RetUser.SignonOK">
> ? ?</input>
> </FORM>
>
> * My Ruby/Mechanize Code
>
--------------------------------------------------------------------------------
> require ''rubygems''
> require ''mechanize''
> require ''mechanize_extns/form_extn.rb''
> require ''logger''
>
> agent = WWW::Mechanize.new{|a| a.log = Logger.new(STDERR) }
>
> # Step 1 - Navigate to Login Page
> page =
agent.get(''https://www.anz.com/INETBANK/login.asp'')
> if !page.forms[1] || (page.forms[1].name != "loginForm")
> ?puts "ERROR: At login form validation failed"
> ?exit
> end
>
> # Step 2 - Login
> login_form = page.forms[1]
> login_form.CorporateSignonCorpId = "xxxx"
> login_form.CorporateSignonPassword = "yyyy"
> page = agent.submit(login_form)
> if !page.forms[0] || (page.forms[0].name != "securityForm")
> ?puts "ERROR: At clicking on continue button after login"
> ?exit
> end
>
> # Step 3 - Click Continue to Main Page ?(** WHERE ISSUE OCCURS **)
> continue_form = page.forms[0]
> # work around 1 - the bank''s input tage of type image does not
seem to
> get handled, so add a button in
> ?
?continue_form.add_field!("Action.RetUser.SignonOK","Proceed to
> Internet Banking") ?# Don''t see effect of this in log trace?
> ?
?continue_form.add_field!("Action.RetUser.SignonOK.x","23")
> ? ? ? ? ? ? ? ? ? ? ? ? ? # Don''t see effect of this in log trace?
> ?
?continue_form.add_field!("Action.RetUser.SignonOK.y","7")
> ? ? ? ? ? ? ? ? ? ? ? ? ? ?# Don''t see effect of this in log
trace?
> # work around 2 - the relative URI in the bank form''s action does
not
> seem to get handled, so add the front part on
> ? ?continue_form.action = "https://banking2.anz.com/IBAU/" +
> continue_form.action
> page = agent.submit(continue_form)
> if !page.forms[1] || (page.forms[1].name != "formAll1")
> ?puts "ERROR: At clicking on continue button after login
> ==================================" ? ?# <== FAILED HERE WITH
SESSION
> ENDED IN REPONSE
> ?puts page.content
> ?puts "ERROR: At clicking on continue button after login
> =================================="
> ?exit
> end
>
>
> # Step Final - Logout
> puts "Completed!"
>
--------------------------------------------------------------------------------
>
>
>
> Any other info that would help you out helping me out?
Your code looks fine.  Your best bet is to mess with the post form
until the post parameters match the browser.  Once you get the post
parameters to match the browser, if it still doesn''t work, then make
sure the headers match the browser headers.

Hope that helps.

-- 
Aaron Patterson
http://tenderlovemaking.com/

2009/8/30 Aaron Starr <astarr at wiredquote.com>:
>
> You should look into Charles:?http://www.charlesproxy.com/
>
> This will allow you to watch the encrypted traffic from your browser, so
you
> can see exactly what cookies, parameters, etc., are being passed back and
> forth. I''d be a little surprised if the login for a bank consisted
of a
> single POST transaction, since in my experience these things tend to
consist
> of multiple redirects and cookies set through javascript and other
nonsense.
> Aaron
>
Thanks - what about capturing the HTTP traffic from my Ruby/Mechanize
code when I run this?  I don''t suppose "charlesproxy" can
help here.
My thoughts re using the same tool for both the browser (successful)
and ruby/mechanize (unsuccessful).   I did already use firefox LIve
HTTP Headers for the browser (successful) case (see output in my
post).

Actually re login this seemed to work fine.  It''s just on the simple
subsequent "click here to continue to the main page" form where
I''ve
got the problem.  I guess this is the first form POST after I am
logged in and either I (or Mechanize code) is missing something?

2009/8/30 Aaron Patterson <aaron.patterson at
gmail.com>:
> Your code looks fine. ?Your best bet is to mess with the post form
> until the post parameters match the browser. ?Once you get the post
> parameters to match the browser, if it still doesn''t work, then
make
> sure the headers match the browser headers.
Thanks heaps AaronP - I''ll try using the hash based approach you
suggested and try to get things matching.

BTW - did my thoughts re Mechanize seeming not quite supporting the
INPUT TYPE="Image" tag make sense to you?

Thanks - what about capturing the HTTP traffic from my
Ruby/Mechanize
> code when I run this?  I don''t suppose "charlesproxy"
can help here.

I''m not sure why not?

But, if you''re happy with your existing debugging tools, then by all
means
just work on getting the mechanize session to match the browser session, as
Aaron P. suggested. If the POST parameters, cookies, and other headers are
the same, then the browser response will be the same.

Aaron
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/mechanize-users/attachments/20090829/45826d67/attachment.html>

2009/8/30 Aaron Patterson <aaron.patterson at
gmail.com>:
> ?Once you get the post
> parameters to match the browser, if it still doesn''t work, then
make
> sure the headers match the browser headers.
>
Arrrrrrr - I used the hash based approach and those parameters seem to
be now in based on the Mechanize debug log, but this didn''t fix it...

BUT....I did review the various header info and I see in my Mechanize
log the cookie is missing!  What''s the approach re using Mechanize and
cookies?  I mean on the main Mechanize doco page it says "Mechanize
automatically stores and sends cookies".   So in this case do I need
to manually do something to add the code in?  (i.e. before I initiate
the post via "page = agent.submit(continue_form)")   Any pointers re
how to clarify what I need and the syntax here?

thanks

[As an aside I was assuming the jsession_id present would have been
what would have been used?  I''ll have to brush up on jession_id usage
versus HTTP sessions and how session state is maintained here.]

If the site uses javascript to create the cookie, then Mechanize won''t
handle it. It handles cookies that are set in the normal way (i.e., through
a header in the response).

I''ve used code similar to this in the past to set a cookie before
submitting:

  def add_cookie(uri, cookie)
    uri = URI.parse(uri)
    WWW::Mechanize::Cookie.parse(uri, cookie) do |cookie|
      @web_agent.cookie_jar.add(uri, cookie)
    end
  end

[...]
    add_cookie(''https://www.someurl.com/'',
''CookieName=some-cookie-value;
path=/; domain=.someurl.com'')
    form.submit


If there''s a more elegant solution, I''d love to know what it
is.

Aaron


On Sat, Aug 29, 2009 at 1:23 PM, Greg Hauptmann <
greg.hauptmann.ruby at gmail.com> wrote:
> 2009/8/30 Aaron Patterson <aaron.patterson at gmail.com>:
> >  Once you get the post
> > parameters to match the browser, if it still doesn''t work,
then make
> > sure the headers match the browser headers.
> >
>
> Arrrrrrr - I used the hash based approach and those parameters seem to
> be now in based on the Mechanize debug log, but this didn''t fix
it...
>
> BUT....I did review the various header info and I see in my Mechanize
> log the cookie is missing!  What''s the approach re using Mechanize
and
> cookies?  I mean on the main Mechanize doco page it says "Mechanize
> automatically stores and sends cookies".   So in this case do I need
> to manually do something to add the code in?  (i.e. before I initiate
> the post via "page = agent.submit(continue_form)")   Any pointers
re
> how to clarify what I need and the syntax here?
>
> thanks
>
> [As an aside I was assuming the jsession_id present would have been
> what would have been used?  I''ll have to brush up on jession_id
usage
> versus HTTP sessions and how session state is maintained here.]
> _______________________________________________
> Mechanize-users mailing list
> Mechanize-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mechanize-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/mechanize-users/attachments/20090829/10156be3/attachment.html>

Thanks - I''ll try this cookie parsing code...the site works fine
without JavaScript.

Btw- so it is the case then that mechanize doesn''t put in the cookie
by default?  I wonder what the mechamize doco refers to supporting
cookies?

Tks again

-- 
Greg
http://blog.gregnet.org/

oh dear

* After clearing cookies in the browser & using HTTP Live Headers &
CharlesProxy I see that I can use my browser successfully without
cookies appears (and I have javascript disabled).   I can''t see how
inserting a cookie will help here therefore.

* The request parameters are now matching (perhaps in a different
order but I assume this doesn''t matter)

*  The request headers seem OK, there are a few that are difference
but I think this would be ok no?
- I have "accept-encoding => gzip,identity" NOT
"gzip,deflate"
- User agent is different of course - I have "user-agent =>
WWW-Mechanize/0.9.3 (http://rubyforge.org/projects/mechanize/)". I
wonder whether I should try to make it match that of firefox?

* The visual text in the page I get back does say things including the
following:
Why Your Session Has Ended
Timing out of sessions
The most likely reason is that you have exceeded your set time limit
for submitting information
You may have:
  Resubmitted a request
  Asked for a page from a banking session that had ended
  Used a web browser that is not fully compatible with this service
What you can do?
  If you came to this page by using a saved bookmark, please remove
this bookmark from your list
  You may need to check the web browser you are using
  Return to the log on page to commence a new Internet Banking session


Any other ideas/techniques to try?   I''m running out of ideas?



Thanks

PS.  If this helps:

******* Successful Post in Firefox - Info from Live HTTP Headers
Plugin ********  (reduced session id size down)
- from Generator Tab -
POST /IBAU/BANKAWAYTRAN;jsessionid=xxxxk%2F
Action.RetUser.SignonOK.x=31&Action.RetUser.SignonOK.y=13&Action.RetUser.SignonOK=Proceed+to+Internet+Banking
- from Headers Tab -
https://banking2.anz.com/IBAU/BANKAWAYTRAN;jsessionid=xxxxk%2F
POST /IBAU/BANKAWAYTRAN;jsessionid=xxxx%2F HTTP/1.1
Host: banking2.anz.com
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US;
rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer:
https://banking2.anz.com/IBAU/BANKAWAY?Action.ANZRetUser.External.SignOn=Y&SecureSubmissionParameter=Y&AppSignonBankId=AUANZ&AppType=corporate&CorporateSignonLangId=001&Country=AU&AffiliateId=00010
Content-Type: application/x-www-form-urlencoded
Content-Length: 109
Action.RetUser.SignonOK.x=31&Action.RetUser.SignonOK.y=13&Action.RetUser.SignonOK=Proceed+to+Internet+Banking
HTTP/1.x 200 OK
Cache-Control: max-age=0
Connection: close
Date: Sun, 30 Aug 2009 01:45:25 GMT
Pragma: no-cache
Content-Type: text/html;charset=ISO-8859-1
Content-Language: en-AU
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Server: Microsoft-IIS/6.0, WebSphere Application Server/5.1
X-Powered-By: ASP.NET
Content-Encoding: gzip
Vary: Accept-Encoding
Transfer-Encoding: chunked

****** unsuccessful post in Mechanize/Ruby - take from Log file ***********
D, [2009-08-30T10:51:03.165628 #5622] DEBUG -- : query:
"Action.RetUser.SignonOK=Proceed+to+Internet+Banking&Action.RetUser.SignonOK.x=23&Action.RetUser.SignonOK.y=7"
I, [2009-08-30T10:51:03.418785 #5622]  INFO -- : Net::HTTP::Post:
/IBAU/BANKAWAYTRAN;jsessionid=xxxx9OF5
D, [2009-08-30T10:51:03.418917 #5622] DEBUG -- : request-header:
accept-language => en-us,en;q=0.5
D, [2009-08-30T10:51:03.418964 #5622] DEBUG -- : request-header:
connection => keep-alive
D, [2009-08-30T10:51:03.419005 #5622] DEBUG -- : request-header: accept =>
*/*
D, [2009-08-30T10:51:03.419046 #5622] DEBUG -- : request-header:
accept-encoding => gzip,identity
D, [2009-08-30T10:51:03.419141 #5622] DEBUG -- : request-header:
content-type => application/x-www-form-urlencoded
D, [2009-08-30T10:51:03.419182 #5622] DEBUG -- : request-header:
user-agent => WWW-Mechanize/0.9.3
(http://rubyforge.org/projects/mechanize/)
D, [2009-08-30T10:51:03.419224 #5622] DEBUG -- : request-header:
referer =>
https://banking1.anz.com/IBAU/BANKAWAY?Action.ANZRetUser.External.SignOn=Y&SecureSubmissionParameter=Y&AppSignonBankId=AUANZ&AppType=corporate&CorporateSignonLangId=001&Country=AU&AffiliateId=00010
D, [2009-08-30T10:51:03.419267 #5622] DEBUG -- : request-header:
accept-charset => ISO-8859-1,utf-8;q=0.7,*;q=0.7
D, [2009-08-30T10:51:03.419308 #5622] DEBUG -- : request-header:
content-length => 108
D, [2009-08-30T10:51:03.419348 #5622] DEBUG -- : request-header: host
=> banking2.anz.com
D, [2009-08-30T10:51:03.419389 #5622] DEBUG -- : request-header:
keep-alive => 300
D, [2009-08-30T10:51:03.563672 #5622] DEBUG -- : Read 882 bytes



thanks again - if you can spot a reason why for the response from
Mechanize I get the "session expired/no longer valid" that''d
be great!




2009/8/30 Greg Hauptmann <greg.hauptmann.ruby at
gmail.com>:
> oh dear
>
> * After clearing cookies in the browser & using HTTP Live Headers &
> CharlesProxy I see that I can use my browser successfully without
> cookies appears (and I have javascript disabled). ? I can''t see
how
> inserting a cookie will help here therefore.
>
> * The request parameters are now matching (perhaps in a different
> order but I assume this doesn''t matter)
>
> * ?The request headers seem OK, there are a few that are difference
> but I think this would be ok no?
> - I have "accept-encoding => gzip,identity" NOT
"gzip,deflate"
> - User agent is different of course - I have "user-agent =>
> WWW-Mechanize/0.9.3 (http://rubyforge.org/projects/mechanize/)". I
> wonder whether I should try to make it match that of firefox?
>
> * The visual text in the page I get back does say things including the
> following:
> Why Your Session Has Ended
> Timing out of sessions
> The most likely reason is that you have exceeded your set time limit
> for submitting information
> You may have:
> ?Resubmitted a request
> ?Asked for a page from a banking session that had ended
> ?Used a web browser that is not fully compatible with this service
> What you can do?
> ?If you came to this page by using a saved bookmark, please remove
> this bookmark from your list
> ?You may need to check the web browser you are using
> ?Return to the log on page to commence a new Internet Banking session
>
>
> Any other ideas/techniques to try? ? I''m running out of ideas?
>
>
>
> Thanks
>


-- 
Greg
http://blog.gregnet.org/

update - Interestingly enough the code starting working and not
throwing the error today.  It seems to be related to how I was running
it Netbeans, as if when I did have a problem and it didn''t work.
Restarting netbeans seemed to have cleared whatever the Netbeans /
Mechanize cache that existed I''m guessing....?

2009/8/31 Greg Hauptmann <greg.hauptmann.ruby at
gmail.com>:
> update - Interestingly enough the code starting working and not
> throwing the error today. ?It seems to be related to how I was running
> it Netbeans, as if when I did have a problem and it didn''t work.
> Restarting netbeans seemed to have cleared whatever the Netbeans /
> Mechanize cache that existed I''m guessing....?
>
BTW - For future reference, thought I should point out I solved the
issue.  There was a hint in the previous code I''d posted:

# work around 2 - the relative URI in the bank form''s action does not
seem to get handled, so add the front part on
   continue_form.action = "https://banking2.anz.com/IBAU/" +
continue_form.action

The work around for the mechanize issue not getting the URL correct, I
had it as "banking2", however in fact what was happening is the site
must be using round robin, and sometimes this would work, but other
times it would be a different URL like "banking1".   Fixed this up and
things worked fine then, i.e. not intermittent issues.    doh!