It might help to try to find the manual case that produces the
''invalidSessionPwd'' error. It sounds like maybe the session
variable
is getting lost. But how the session is tracked varies from app to app.
-Mat
On Feb 9, 2009, at 5:27 PM, Andre Nathan wrote:
> Hello
>
> I''m trying to scrape www.claroideias.com.br. My idea was to build
a
> Shoes interface to their SMS service, with the backend done in
> mechanize. I managed to get to the very last stage of submitting the
> response to the captcha file, but I always get a session error which I
> don''t understand. The form is validated using javascript, and
these
> are some of its return values:
>
> C?digo: 0 - sucessPost
> C?digo: 5 - invalidPwd
> C?digo: 8 - invalidSessionPwd
> C?digo: 9 - invalidSessionCtn
>
> I always get the "invalidSessionPwd" error, no matter what is
written
> in the captcha form, so I believe this is a misuse of mechanize on my
> part. When using firefox, if I write the wrong captcha answer, the
> error returned is "invalidPwd". If before submitting the form I
delete
> my cookies, I get "invalidSessionCtn". I tried running tcpdump
and
> comparing the data from firefox and mechanize, but couldn''t find
any
> difference that could be related to session errors.
>
> The code follows. If someone could give it a try, it would be awesome.
>
> require ''rubygems''
> require ''mechanize''
>
> #
> # Use as:
> #
> # data = {
> # :src_name => ''XXXXXX'',
> # :src_code => ''XX'',
> # :src_phone => ''XXXXXXXX'',
> # :dst_code => ''XX'',
> # :dst_phone => ''XXXXXXXX'',
> # :message => "XXXX"
> # }
> #
> # scraper = Scraper.new(data, "out.dir")
> # captcha = scraper.fetch_captcha # display to the user, etc.
> # scraper.submit_answer(''xxxx'') # the answer
doesn''t really matter,
> always gets error 8
> #
>
> class Scraper
> SMS_MAX_LEN = 136
>
> def initialize(data, out_dir)
> @data = data
> @agent = WWW::Mechanize.new
> @dir = out_dir
> @captcha_page = nil
> end
>
> def fetch_captcha
> path, id, ref = submit_state
> captcha_page = submit_sms(path, id, ref)
> fetch_image(captcha_page)
> end
>
> def submit_answer(answer)
> raise ''Not in captcha page!'' if @captcha_page.nil?
> form = @captcha_page.form_with(:action =>
''DeliveryServlet'')
> form.fields.first.name
> form[form.fields.first.name] = answer
> @agent.get(:url =>
>
''http://clarotorpedoweb.claro.com.br/ClaroTorpedoWeb/scripts/clarotorpedo.jsp''
> ,
> :referer =>
''http://clarotorpedoweb.claro.com.br/ClaroTorpedoWeb/pwdForm.jsp'')
> p form.submit
> end
>
> private
> def submit_state
> url = ''http://www.claroideias.com.br''
> page = @agent.get(url)
> form = page.form_with(:name => ''frmEntrada'')
> act = form.action
> states = form.field_with(:name => ''links'')
> val = states.options.find { |opt| opt.text == ''Rio de
> Janeiro'' }.value
> path, id = val.split(/\|/)
> form[''links''] = val
> form[''txtUrl''] = path
> form[''txtIDLocal''] = id
> form.submit
> ref = "#{url}#{act}?
> txtUrl=#{path}&txtIDLocal=#{id}&gclid=&links=#{val}"
> return path, id, URI.escape(ref)
> end
>
> private
> def submit_sms(path, id, ref)
> url = URI.escape("http://
> www.claroideias.com.br#{path}&idlocal=#{id}")
> page = @agent.get(:url => url, :referer => ref)
> form = page.form_with(:name => ''Main'')
> form[''ddd_para''] = @data[:dst_code]
> form[''telefone_para''] = @data[:dst_phone]
> form[''ddd_de''] = @data[:src_code]
> form[''telefone_de''] = @data[:src_phone]
> form[''nome_de''] = @data[:src_name]
> form[''msg''] = @data[:message]
> form[''caract''] = sms_max_len
> return form.submit
> end
>
> private
> def fetch_image(page)
> image = nil
> page.search(''img[@src]'').each do |img|
> if img[''src''] =~ /^\/ClaroTorpedoWeb\//
> image = img[''src'']
> break
> end
> end
> raise "Couldn''t find captcha" if image.nil?
> FileUtils.mkdir_p(@dir)
> file = File.join(@dir, "#{File.basename(image)}.jpg")
> Net::HTTP.start(page.uri.host) do |http|
> resp = http.get(image)
> File.open(file, "wb") { |f| f.write(resp.body) }
> end
> @captcha_page = page
> return file
> end
>
> def sms_max_len
> len = SMS_MAX_LEN - @data[:src_name].size - @data[:message].size
> raise ''Message too long'' if len < 0
> return len
> end
> end
>
> Thanks in advance,
> Andre
> _______________________________________________
> Mechanize-users mailing list
> Mechanize-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mechanize-users