similar to: Getting elements from a web page

I am trying to get a page which includes a form, but the form is missing from the WWW::Mechanize::Page object. I retrieve it via: page = web_agent.submit(a_different_form) For debugging this problem, I then immediately write the resulting page to two different logs: File.open(''big.html'',''wb'') { |f| f.write(page.body) }

Hello, I have a login form on /login.php which POSTs to /dorf1.php when access is granted and to /login.php when it is denied. require ''rubygems'' require ''mechanize'' agent = WWW:Mechanize.new() login = agent.get("http://server/login.php") form = login.forms.action("dorf1.php") form.fields[2].value = "wronguser" # login

Is there a decent way to add a field to a form before posting it? I haven''t tried using HPricot manipulations just yet, since I can''t ever find really solid docs on hpricot.... Form#[]= doesn''t work because it first searches only pre-existing fields. I''m investigating how to write a patch now. But I thought maybe someone here might have an idea.

Hi all, I am so excited to use mechanize! It has opened a whole new world of projects for me :) I am trying to login into the Gmail.com server, as described in http://schf.uc.org/articles/2007/02/14/scraping-gmail-with-mechanize-and-hpricot but am running into a few issues... irb(main):010:0> page = agent.submit form EOFError: end of file reached from

This is in windows .. So from irb, I can load hpricot ok. (ruby 1.8.7) C:\Users\Laurence\rails-play\plug-test>irb irb(main):001:0> require ''hpricot'' LoadError: no such file to load -- hpricot from (irb):1:in `require'' from (irb):1 irb(main):002:0> require ''rubygems'' => true irb(main):003:0> require

Hi, This is my first post, I wanted to ask a quick question about "Bad Form" -- rather, am I doing something with Mechanise I shouldn''t be doing? Here''s the long and short of it - I am using Mechanize with Cucumber and RSpec to test a *ahem* Perl XML web service.. talk about technology soup, but I have it working quite well. I have an example project on GitHub that

hi - i have strings that i need to extract keywords from. the string might have html tags, urls, etc. i need to extract the keywords from the string. i imagine i''m not the first guy to have to tackle this problem. is there a gem i can use or anyone have any ideas how to approach this? thanks, dino

So I found myself wanting to call Mechanize#get with a hash for arguments like this: WWW::Mechanize.new(''http://api.flickr.com/services/rest/'', {:method => ''flickr.auth.getFrob''... }) Granted, it looks like this isn''t supported but it led me to what looks like a bug. Namely that get calls fetch_page(abs_uri, request, cur_page, &block)

Hi, I have been stumped by this error Hpricot is generating in my rails app. The error looks like this: $ script/console Loading development environment (Rails 2.0.2) >> str = ''<p>lorem ipsum <a href="http://dolor.com/">dolor</a> sit amet.</p>'' => "<p>lorem ipsum <a href=\"http://dolor.com/\">dolor</a>

I found an interesting page today that I was trying to script against. The server returns a 404 with content and the page just works normally in firefox despite the 404. Mechanize raises an exception on it though. I''m working on a test case now.

Hello to the list and thanks to Aaron for the cool software. I''ve been fooling around with Mechanize and Hpricot for a couple of days and from the docs I''ve read, the following code SHOULD work but doesn''t. I''ve tried the same code on a couple of different sites and I get the same exception for each. Any pointers or suggestions are appreciated.

Here''s what I get when I try to use the console with the current facebooker: First I make a simple app: rails foo -d mysql cd foo script/plugin install git://github.com/mmangino/facebooker.git Then I go to config/environment.rb and change the rails version to ''2.2.2'' Then I run the console: script/console and I get: [17:56][herdrick:~/rails/foo]$ script/console

Hi there, Has anyone tried acts_as_ferret with Edge Rails and Mongrel? When I install the plugin to a project that has Edge Rails frozen, and the Mongrel gem installed, I can''t start the server. There''s no error, it just doesn''t start. I''ve used acts_as_ferret in the past with WEBrick, and stable Rails releases without a hitch. If I remove the

I read the 13th recipe in the book "Rails Recipes" from Chad Fowler. I install with gem the gruff-library (gem install gruff). After the installation, I download the "rmagick-win32-1.13.0_IM-6.2.9-3.zip" file and install it on my win32-machine. It was successfully! So, I create a controller with script/generate. In a method, which I wrote after the creation, I''ll make

I noticed an interesting problem today when scripting against a web app. The application contained a link in it that used %20 instead of spaces. After running mechanize through the Charles debugging proxy I found that mechanize was converting %20 to %2520 (double escaping the %20). This appears to happen under both 0.5.4 and 0.6.0. Here''s a simple set of files that demonstrate

Hi. Lighttpd''s merrily serving Rails on OS X for me, but I can''t for the life of me get script/console to run. I get, always: /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:18:in `require__'': No such file to load -- readline (MissingSourceFile) (And then a chain of errors leading from that point...) I followed Jason''s manuals.textdrive

Hi guys, I wanted to fetch some info from webpage and use it in my db. I read hpricot is one way to do it. But the problem is I''m unable to use it with my app. For some reason after successfully installing hpricot it fails when I try to use it in the rails console. I''ve been trying to fix this for the last four-five days. I''m stumped ! I tried re-installing everything

I''m trying to use mechanize against a site that has four fields in the form. However, those four fields have to be filled in order. So putting something in field one, populates the second field drop down. So I''m thinking that I''ll probably have to call the page multiple times? What sort of things should I be doing to figure out how to interact with this page.

Heyas :) I''m wondering how do I do to send my custom HTTP headers when posting a form. agent.set_headers is a private method and I don''t know how to get a reference to HTTPHeaders to use add_field and such. Since my request is a form, I''m sending it using agent.submit(form). Any hints? Bonus question: I would like to upload a file to a REST webservice, but I

I haven''t spent the time to get a proper test case for this yet, but there appears to be a bug in the basic_auth code for mechanize 0.6.9. I''ve attached a CSV (from Charles) that illustrates the problem. Basically when running with basic_auth, there''s a failed request that''s followed up by a successful request. That last POST is a agent.submit(form)