On Mon, Jul 23, 2007 at 10:16:59AM -0800, barsalou wrote:> I''m trying to use mechanize against a site that has four fields in the > form. However, those four fields have to be filled in order. > > So putting something in field one, populates the second field drop down. > > So I''m thinking that I''ll probably have to call the page multiple times? > > What sort of things should I be doing to figure out how to interact > with this page.How does the browser do it? Are the fields populated via Javascript?> > I''m currently just using trial and error, but thought I might tap the > experience here to see if anyone has run into this sort of site before. > > Thanks for any guidance.Typically when I''m scripting a site, I''ll try to follow what the browser does as closely as possible. If I''m not sure exactly what the browser is doing I''ll open up LiveHTTPHeaders. http://livehttpheaders.mozdev.org/ Hope that helps. -- Aaron Patterson http://tenderlovemaking.com/
I''m trying to use mechanize against a site that has four fields in the form. However, those four fields have to be filled in order. So putting something in field one, populates the second field drop down. So I''m thinking that I''ll probably have to call the page multiple times? What sort of things should I be doing to figure out how to interact with this page. I''m currently just using trial and error, but thought I might tap the experience here to see if anyone has run into this sort of site before. Thanks for any guidance. Mike B. ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.
Quoting Aaron Patterson <aaron at tenderlovemaking.com>:> On Mon, Jul 23, 2007 at 10:16:59AM -0800, barsalou wrote: >> I''m trying to use mechanize against a site that has four fields in the >> form. However, those four fields have to be filled in order. >> >> So putting something in field one, populates the second field drop down. >> >> So I''m thinking that I''ll probably have to call the page multiple times? >> >> What sort of things should I be doing to figure out how to interact >> with this page. > > How does the browser do it? Are the fields populated via Javascript?Yes. This seems like the way things are being done. Does that make this an impossible task? Should I handle a site with this much JS differently than I would other sites?> >> >> I''m currently just using trial and error, but thought I might tap the >> experience here to see if anyone has run into this sort of site before. >> >> Thanks for any guidance. > > Typically when I''m scripting a site, I''ll try to follow what the browser > does as closely as possible. If I''m not sure exactly what the browser > is doing I''ll open up LiveHTTPHeaders. > > http://livehttpheaders.mozdev.org/ > > Hope that helps. >I''ll take a look at that. Thanks for the suggestion. ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.
On Jul 23, 2007, at 3:01 PM, barsalou wrote:> Yes. This seems like the way things are being done. Does that make > this an impossible task? Should I handle a site with this much JS > differently than I would other sites?I''ve used mechanize for heavy JS sites (like hotmail) by reverse engineering just enough of the javascript to make the appropriate HTTP call. I usually work out the logic then port it to ruby so I can use the same basic structure as the main website. Not exactly easy, but doable. In addition to LiveHTTPHeaders I also use TamperData [1], Firebug [2] and Charles [3] for this sort of work. -Mat [1] https://addons.mozilla.org/en-US/firefox/addon/966 [2] https://addons.mozilla.org/en-US/firefox/addon/1843 [3] http://www.xk72.com/charles/
Quoting Mat Schaffer <schapht at gmail.com>:> On Jul 23, 2007, at 3:01 PM, barsalou wrote: >> Yes. This seems like the way things are being done. Does that make >> this an impossible task? Should I handle a site with this much JS >> differently than I would other sites? > > I''ve used mechanize for heavy JS sites (like hotmail) by reverse > engineering just enough of the javascript to make the appropriate > HTTP call. I usually work out the logic then port it to ruby so I > can use the same basic structure as the main website. Not exactly > easy, but doable. > > In addition to LiveHTTPHeaders I also use TamperData [1], Firebug [2] > and Charles [3] for this sort of work. > > -Mat > > [1] https://addons.mozilla.org/en-US/firefox/addon/966 > [2] https://addons.mozilla.org/en-US/firefox/addon/1843 > [3] http://www.xk72.com/charles/Thanks. This is helpful. Mike B. ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.