similar to: Submitting a form sends a file. How do I save it?

Hi, I''m working to scrape and save some ebooks. Mechanize has been wonderful so far. The link I''m having trouble with is this one. http://www.webscription.net/SendZip.aspx?SKU=0671578499&ProductID=379&format=H When I click that in the browser it saves it to a file named H_1632.zip. How do I get that name from the page. I suspect to save this to a file I would just do

Hi, Does anyone have the formula for getting logged into LinkedIn? Here''s my current attempt: require ''rubygems'' require ''mechanize'' agent = WWW::Mechanize.new home_page = agent.get(''http://www.linkedin.com'') signin_page = agent.click home_page.links.text(''Sign in'') puts "\nSIGNIN PAGE"

When I click a link to download a file, a pop up window comes up to save the file. Is there a way to enter a file name and click the submit button with mechanize? Thanks, Jeff -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mechanize-users/attachments/20070712/558f7018/attachment.html

Hi, I''m having some trouble downloading a .csv file from a particular website. The file isn''t part of a url, you need to click on a link in order to get the file sent. I don''t know how to get mechanize to correctly identify that. Here is the link to the file I''m trying to retrieve: <td style="vertical-align: bottom; text-align: center;">

Hi, I''m using Mechanize, and I''ve developed a lot of code around it. I''d like to be able to check the Etag header during a get to see if the page has changed, as well as some other http header information. Can I do that without hacking Mechanize myself? Does anyone have any examples of how to do this? William

Hello to the list and thanks to Aaron for the cool software. I''ve been fooling around with Mechanize and Hpricot for a couple of days and from the docs I''ve read, the following code SHOULD work but doesn''t. I''ve tried the same code on a couple of different sites and I get the same exception for each. Any pointers or suggestions are appreciated.

Hi all, I''ve been using mechanize for a while and it rocks. Docs are pretty clear and so far I''ve been able to do it on my own. However, I''m stuck in a weird situation in a script to download my contact list from hotmail. I''ve used Firebug to check all urls, and tested it by hand while logged in via browser. Even in the script everything works well until the

Hey WWW::Mechanize users, I''ve been using Mechanize for about 3 or 4 months now, and I''m rather enjoying it. I''ve done scraping in PHP & Python, starting with raw text and HTTP requests, moving up to regex and Curl, and finally landed myself here, making me much more productive when scraping pages. At the moment, I''m in the middle of a decent sized

version 0.62 I''ve got a form that''s giving me trouble (server reports invalid session ID). Unfortunately, the form is submitted using SSL so I can''t use a sniffer to see the differences between the ruby mechanize version and the perl mechanize version that is working. Is there a way to change the form action within mechanize? Or a way to display the request headers?

Hi all, I''ve a problem when submitting a form : the URL seems to be invalid. I really don''t see the way I should try to solve that problem. I searched on the web, but didn''t found any kind of that problem. Here is the code : require ''mechanize'' agent = WWW::Mechanize.new page =

All, Another easy question. In Hpricot, on a doc that I am using, I can do a .to_html method and retrieve the entire page. However, this doesn''t seem to work in Mechanize. My goal is to the text of the page and put it into a database to make it searchable with ferret (using the acts_as_ferret plugin in Rails). Does anyone have a good suggestion short of iterating over the entire

Hello I am trying to fill in a form using WWW::Mechanize. I can fill in 2 of the 3 fields but one is giving me a problem. The name of this field is "name" when I use the following bit of code, it seems to change the name of the form, not the value of the field. The part of original page after pretty printing {forms #<WWW::Mechanize::Form {name "wp_pers_form"}

I just started using Mechanize, and started using Ruby about thirty seconds before that, but one of the sites I''m scraping does a redirect on form submission to a badly-formed relative URL: index.cfm?action=bing&bang=boom=1|a=|b=|c= (etc.) Interestingly, Mechanize 0.6.2 handled this OK, but in 0.6.3 this causes a URI::InvalidURIError exception from URI.parse() in to_absolute_uri

I am new to Mechanize and was wondering if there was a built-in method to get the elements that are on the page that are not part of a form. A couple of examples would be my banking site lists my entries and I want them to go into an array so that I can handle them. Or another site I use, does some categorization for me and I would like to manipulate it and present it differently to a user.

Hi, I''ve recently had some trouble setting form values in my code - for example: require ''rubygems'' require ''mechanize'' def save(pagename, content, editsummary = '''') agent = WWW::Mechanize.new agent.user_agent = ''Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)'' page =

Trying to submit a form is resulting in the following error: >> page = agent.submit(f, f.buttons.first) query: "action=start&quick=Y&clientApp=0&clientID=MCO" Net::HTTP::Post: /dealerApply/controller.jsp using cookie: JSESSIONID=GJ4NCvGJDTlrvrKsLw17hZ3FnlXFdVkvB5hh1jLkHKcH4Tn27371!-538794652 request-header: accept-language => en-us,en;q0.5 request-header: connection

All, Been a while since I''ve "mechanized" and I''m running into an idiot simple problem. I''m looking for more detailed examples to refresh my memory. My question is how to get a list of links off of a page based on information in the img src attribute, specifically if the img tag contains a "mp3.gif/jpg". I, for the life of me, can''t seem

Hey all, Found a email thread from Jan 2007 discussing the inability to set request headers (like ETag and If-Modified-Since) through the API, and this is something that''s bothering me a bit. Currently the "way" to do this is to subclass Mechanize and override set_headers(). That seems fine for headers that you''d like to send in every request or for classes of request,

Hi all, Trying to figure out a quick way to do something. I have a yahoo groups list, and it has listings that look like this: </td></tr> <tr><td colspan="2" class="ygrp-g"> <span class="ygrp-nowrap"> <em><a href="/group/freecycledc/">freecycledc</a></em> </span> <a

Hi, I''m interested in using Mechanize as a way around manual multipart form creation. But the API is really centered around the idea of a form built from a page. I''d love to see a syntax like: response = agent.post(''http://www.mydomain.com/upload'', :image => File.new(file), :title => "My image", :description => "an image