similar to: Quick and Easy Question

All, Another easy question. In Hpricot, on a doc that I am using, I can do a .to_html method and retrieve the entire page. However, this doesn''t seem to work in Mechanize. My goal is to the text of the page and put it into a database to make it searchable with ferret (using the acts_as_ferret plugin in Rails). Does anyone have a good suggestion short of iterating over the entire

I am new to Mechanize and was wondering if there was a built-in method to get the elements that are on the page that are not part of a form. A couple of examples would be my banking site lists my entries and I want them to go into an array so that I can handle them. Or another site I use, does some categorization for me and I would like to manipulate it and present it differently to a user.

Hi all, Trying to figure out a quick way to do something. I have a yahoo groups list, and it has listings that look like this: </td></tr> <tr><td colspan="2" class="ygrp-g"> <span class="ygrp-nowrap"> <em><a href="/group/freecycledc/">freecycledc</a></em> </span> <a

Hello to the list and thanks to Aaron for the cool software. I''ve been fooling around with Mechanize and Hpricot for a couple of days and from the docs I''ve read, the following code SHOULD work but doesn''t. I''ve tried the same code on a couple of different sites and I get the same exception for each. Any pointers or suggestions are appreciated.

Hey everyone, I''ve been refactoring Mechanize for an 0.7.0 release. Basically I''m trying to clean the code up and there are a few features that I think are unnecessary, but I would like to ask people first. 1) REXML as a parser. I want to remove support for REXML. I don''t use it. Hpricot seems to do everything I need. 2) 1.8.2 thru 1.8.4 support

mechanize version 0.6.6 has been released! http://mechanize.rubyforge.org/ The Mechanize library is used for automating interaction with websites. Mechanize automatically stores and sends cookies, follows redirects, can follow links, and submit forms. Form fields can be populated and submitted. Mechanize also keeps track of the sites that you have visited as a history. Changes: =

Hi, I''m using Mechanize, and I''ve developed a lot of code around it. I''d like to be able to check the Etag header during a get to see if the page has changed, as well as some other http header information. Can I do that without hacking Mechanize myself? Does anyone have any examples of how to do this? William

I knew that hpricot was a dependency, but hoe and rubyforge....is this right? Version 0.6.9 for ruby Mike B. ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

Is there a decent way to add a field to a form before posting it? I haven''t tried using HPricot manipulations just yet, since I can''t ever find really solid docs on hpricot.... Form#[]= doesn''t work because it first searches only pre-existing fields. I''m investigating how to write a patch now. But I thought maybe someone here might have an idea.

Hello all, I am trying to click image maps ("area" nodes), and while Perl''s mechanize treats "a", "area", "frame", "iframe" and "meta" tags as links, it seems to me that Ruby''s Mechanize thinks only "a" tags can be links (at least my little experiment, namely: page.links.each {|link| puts link.node.name}

I have a Watir script that I would like to convert to a Mechanize script. The watir code looks like this require ''watir'' include Watir def test ie = IE.new ie.goto(''http://cpref.gsm.com/inter.asp?r=8084'') ie.text_field(:name, ''inter_string'').set(''Potassium'') ie.button(:id, ''image1'').click

I got this in one of my pages, and I''m not exactly sure what it means. ideas? Mike B. ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

I''ve been using Mechanize for a project that i''ve been working on, but this is the first time i''m having to use forms (scraping previously). So, after i fill out the form, when I hit submit, it sends me information in the form of a text file to download. For the life of me, I can''t see how to get access to it. When clicking on a link, you can put a

Greetings, I''m using Mechanize to scrap dozens of pages and have noticed the size of my ruby process keeps growing. I set Mechanize.max_history to 0 with no effect on the memory use. I wrote a little test to show the objects left on the heap after mechanizing a single page and then doing a garbage collection. Sample list appended below. I can supply the test code if it helps.

Hello, I am using Mechanize to post a form to a website. When I do this by hand in my browser the response takes about 35s to come back (it''s a long page full of tables and graphics). When I do this with Mechanize, the server starts to respond and then appears to hang. The obvious conclusion is that my code is wrong but I am reasonably sure that I haven''t altered it

How do you make get the page when using basic_auth? Below is code, I am getting a 401 error. I am not sure if the basic _auth comes before the agent.get. I will appreciate any help. require ''mechanize'' require ''logger'' agent = WWW::Mechanize.new {|a| a.log = Logger.new(STDERR) } page = agent.get(''https://brewx.qualcomm.com/developer'' )

Hello I am trying to fill in a form using WWW::Mechanize. I can fill in 2 of the 3 fields but one is giving me a problem. The name of this field is "name" when I use the following bit of code, it seems to change the name of the form, not the value of the field. The part of original page after pretty printing {forms #<WWW::Mechanize::Form {name "wp_pers_form"}

Hey Aaron, I''m just thinking out loud here, but have you considered the possibility of using the Rhino [1] library to implement javascript support in mechanize? It''d create a jruby dependency for that feature, but still. Just thought I''d bring it up while I was thinking about it. -Mat [1] http://www.mozilla.org/rhino/

version 0.62 I''ve got a form that''s giving me trouble (server reports invalid session ID). Unfortunately, the form is submitted using SSL so I can''t use a sniffer to see the differences between the ruby mechanize version and the perl mechanize version that is working. Is there a way to change the form action within mechanize? Or a way to display the request headers?

hi, a potential mechanize user here. i''ve looked at the API but it''s not clear what capacity there is to arbitrarily set the headers - does mechanize allow for this? mechanize is clearly great for web browsing, but i also need to construct get and post requests from scratch. i''ve looked at net:http, libcurl, openuri, but mechanize seems simpler and more functional than