similar to: How to execute time consuming code

Hello all, Yet another "don''t reinvent the wheel" n00b question: Is there an ''official'' page hit counter plugin/code snippet/whatever? I would need something like this: - It is possibility to set a constant timespan (say 5 hours). - On every page hit, save away @request.remote_ip and Time.new - The counter gets incremented only if the same IP was here

Hi there Does anyone know if Amazon.com has any sort of server side script that tries to block scraping activities? I first noticed that if I didn?t change the agent alias, it would fetch a page exactly like the normal one, but without the intial search field(maybe a silly way to prevent scraping). Then after it, I changed to some other alias, and submit a search. I got the result page as

Hi: I am looking for a good hands on web mashup tutorial? or books for that matter.. Google gives me bits and pieces of the info but not any step by step or hands on tutorial.. Thanks a bundle -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk"

Hello all, Maybe the $subj is a little bit weird (i.e. apples vs oranges) but it is a situation i am facing at the moment: We are developing a small web site which will be mostly a CMS (more or less) and my colleagues is arguing for drupal, and i am for RoR. Maybe i can formulate the question in a different way: when to use a CMS (not necessarily drupal but e.g. Radiant CMS) and when to use Ruby

Good afternoon, I''ve two quick questions for the Ruby/Rails community. 1) Do you use Linux for Ruby or Rails development? 2) If you use Linux, which distro do you normally use to develop? Thank you for your input. :-) -- Posted via http://www.ruby-forum.com/.

Hi, I want to get the details from a table which is displayed on a Web page into the Database. How to read that Web page ? I mean if there is table Student details like Roll no., name etc. And I want to read that table & store it into my database in table My_Student. How to do this? --~--~---------~--~----~------------~-------~--~----~ You received this message because you are

Hey all, maybe I am missing the obvious again, but I can not submit the form on buy.com: require ''rubygems'' require ''mechanize'' agent = WWW::Mechanize.new agent.user_agent_alias = ''Mac Safari'' page = agent.get("http://www.buy.com") search_form = page.forms.with.name("searchbox").first p search_form.buttons the last

Hi all, I need to do some screen scraping from my rails app. Given an ethernet (MAC) adress, I scrape results from an internal web page that returns location and hostname. How can I cache the result from that screen scraping as to be polite to the scrapee? I would like to expire the results daily. In perl, I would use Cache::File. Can I use rails caching for this? What''s the best

Hi List, I am trying to leverage my knowledge of R in trying to use it for tasks that may not make R the best choice for these tasks. I wish to automate a web scraping task, which requires a multi-step procedure: 1) log in to a website 2) Go to a particular page 3) From the drop down menu, click on a particular link 4) From the tabulated data presented, choose relevant information based on a

Hey WWW::Mechanize users, I''ve been using Mechanize for about 3 or 4 months now, and I''m rather enjoying it. I''ve done scraping in PHP & Python, starting with raw text and HTTP requests, moving up to regex and Curl, and finally landed myself here, making me much more productive when scraping pages. At the moment, I''m in the middle of a decent sized

Forgive me if this has been addressed somewhere, but I have searched and can''t come up with anything. I am basically trying to distribute several web page scraping tasks among different threads, and have the results from each added to an Array which is ultimately returned by the backgroundrb worker. Here is an example of what I''m trying to do in a worker method: pages =

Hi there Does anyone know if Amazon.com has any sort of server side script that tries to block scraping activities? I first noticed that if I didn?t change the agent alias, it would fetch a page exactly like the normal one, but without the intial search field(maybe a silly way to prevent scraping). Then after it, I changed to some other alias, and submit a search. I got the result page as

hi all, I'm working on scrapping some website data to build a database. Under most cases, I can use package XML to get the dataset. However, some of the website doesn't give a explicit address of the downloaded tables. To be more specific, for example, I'm interested in the website http://ets.aeso.ca/ The data we are scraping is the "Pool Weekly Summary" under the

I''m helping with a gem that is going to published under the contentfree project on rubyforge (http://rubyforge.org/projects/contentfree/). The gem is called "blackbook" and basically it will go and fetch your contacts from the major webmail providers. So far Gmail, Yahoo!, and MSN have been completed. We are trying to finish up with fetching contacts from AOL Webmail. However

This is long overdue (0.2.8 is out for about a week already), but anyway, here we go: ============ What''s this? ============ scRUBYt! is a very easy to learn and use, yet powerful Web scraping framework based on Hpricot and mechanize. It''s purpose is to free you from the drudgery of web page crawling, looking up HTML tags, attributes, XPaths, form names and other typical

I am scraping data from a web page using XML (excellent package BTW - that's scraping data the easy way!). So far, I've got the code: tables <- readHTMLTable(theurl) rhf <- tables$tabResHistFull div1 <- rhf[which(rhf$V1=="Div ps"),] div1 which is giving me the result:        V1 V2    V3    V4    V5    V6    V7          V8    V9   V10   V11   V12   V13   V14  V15 15

Hi, if i am not mistaken 2.4 has a servlet which provides json output for active mounts and stats. You can check it out at http://your host:8000/status-json.xsl Just upgrade your version and you should be fine. Hth In a search for access to various ways to interact with Icecast via Python, I ran across this: http://xiph.org/~brendan/pyshout2/pyshout2-0.0.1.tar.gz Which is now dead (404). I

I am doing a research on World Bank (WB) projects on developing countries. To do so, I am scraping their website in order to collect the data I am interested in. The structure of the webpage I want to scrape is the following: 1. List of countries the list of all countries in which WB has developed projects<http://projects.worldbank.org/country?lang=en&page=> 1.1. By clicking on a

Hello. I am trying to find a way to retrieve data from Harvard Dataverse website. I usually don't have problem in web-scraping data but the problem here is that there are a bunch of data formats such as .tab, .7z and so and I just can't find a way to retrieve the data I am interested in woth an unique solution. Any hint? [[alternative HTML version deleted]]

I'm still having trouble installing the lme4 package on RHEL 3. I've asked this list and it seems my problem is not universal. Brian Ripley indicated that the problem was with recognising the Matrix package, even though I've taken care to get the most recent versions of Matrix and lme4. It seems to me that the problem arises because the computer has a site installation of R-2.3.1