Hi, I would like to download data from below page directly onto R. http://www.nseindia.com/live_market/dynaContent/live_watch/equities_stock_watch.htm Could you please assist me how can I do that programmatically. Thanks for your time.
This is not a simple question. The data are in an html-formatted web page. You must "scrape" the html for the data and read it into an R table (or other appropriate R data structure). Searching (the web) on "scrape data from html into R" listed several packages that claim to enable you to do this "easily". Choose what seems best for you. You should also install and read the documentation for the XML package, which is also used for this purpose, though those you find above may be slicker. Disclaimer: I have no direct experience with this. I'm just pointing out what I believe are relevant resources. Cheers, Bert Bert Gunter "Data is not information. Information is not knowledge. And knowledge is certainly not wisdom." -- Clifford Stoll On Tue, Aug 25, 2015 at 11:10 AM, Christofer Bogaso <bogaso.christofer at gmail.com> wrote:> Hi, > > I would like to download data from below page directly onto R. > > http://www.nseindia.com/live_market/dynaContent/live_watch/equities_stock_watch.htm > > Could you please assist me how can I do that programmatically. > > Thanks for your time. > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.
Looks like you can get what you need from http://www.nseindia.com/homepage/Indices1.json on that page. On Tue, Aug 25, 2015 at 2:23 PM, Bert Gunter <bgunter.4567 at gmail.com> wrote:> This is not a simple question. The data are in an html-formatted web > page. You must "scrape" the html for the data and read it into an R > table (or other appropriate R data structure). Searching (the web) on > "scrape data from html into R" listed several packages that claim to > enable you to do this "easily". Choose what seems best for you. > > You should also install and read the documentation for the XML > package, which is also used for this purpose, though those you find > above may be slicker. > > Disclaimer: I have no direct experience with this. I'm just pointing > out what I believe are relevant resources. > > Cheers, > Bert > Bert Gunter > > "Data is not information. Information is not knowledge. And knowledge > is certainly not wisdom." > -- Clifford Stoll > > > On Tue, Aug 25, 2015 at 11:10 AM, Christofer Bogaso > <bogaso.christofer at gmail.com> wrote: >> Hi, >> >> I would like to download data from below page directly onto R. >> >> http://www.nseindia.com/live_market/dynaContent/live_watch/equities_stock_watch.htm >> >> Could you please assist me how can I do that programmatically. >> >> Thanks for your time. >> >> ______________________________________________ >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.
I agree that this is a tricky task... even more so than using
a"scraping" package because the page is built dynamically. This will
take someone with skills in multiple web technologies to decipher the web page
scripts to figure out how to manipulate the server to give you the data, because
it isn't actually in the web page.
---------------------------------------------------------------------------
Jeff Newmiller The ..... ..... Go Live...
DCN:<jdnewmil at dcn.davis.ca.us> Basics: ##.#. ##.#. Live
Go...
Live: OO#.. Dead: OO#.. Playing
Research Engineer (Solar/Batteries O.O#. #.O#. with
/Software/Embedded Controllers) .OO#. .OO#. rocks...1k
---------------------------------------------------------------------------
Sent from my phone. Please excuse my brevity.
On August 25, 2015 11:23:26 AM PDT, Bert Gunter <bgunter.4567 at
gmail.com> wrote:>This is not a simple question. The data are in an html-formatted web
>page. You must "scrape" the html for the data and read it into an
R
>table (or other appropriate R data structure). Searching (the web) on
>"scrape data from html into R" listed several packages that claim
to
>enable you to do this "easily". Choose what seems best for you.
>
>You should also install and read the documentation for the XML
>package, which is also used for this purpose, though those you find
>above may be slicker.
>
>Disclaimer: I have no direct experience with this. I'm just pointing
>out what I believe are relevant resources.
>
>Cheers,
>Bert
>Bert Gunter
>
>"Data is not information. Information is not knowledge. And knowledge
>is certainly not wisdom."
> -- Clifford Stoll
>
>
>On Tue, Aug 25, 2015 at 11:10 AM, Christofer Bogaso
><bogaso.christofer at gmail.com> wrote:
>> Hi,
>>
>> I would like to download data from below page directly onto R.
>>
>>
>http://www.nseindia.com/live_market/dynaContent/live_watch/equities_stock_watch.htm
>>
>> Could you please assist me how can I do that programmatically.
>>
>> Thanks for your time.
>>
>> ______________________________________________
>> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
>______________________________________________
>R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.
If there's no api available, I would use selenium to grab what I need and pipe it to R. Let me know if you need further assistance. Cheers! -- H On Aug 25, 2015 11:12 AM, "Christofer Bogaso" <bogaso.christofer at gmail.com> wrote:> Hi, > > I would like to download data from below page directly onto R. > > > http://www.nseindia.com/live_market/dynaContent/live_watch/equities_stock_watch.htm > > Could you please assist me how can I do that programmatically. > > Thanks for your time. > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >[[alternative HTML version deleted]]
Hello, There might be a problem: > url <- "http://www.nseindia.com/live_market/dynaContent/live_watch/equities_stock_watch.htm" > readLines(url) Error in file(con, "r") : cannot open the connection In addition: Warning message: In file(con, "r") : cannot open: HTTP status was '403 Forbidden' So I've downloaded the csv file with the data, but that's not programmatically. Rui Barradas Em 25-08-2015 19:10, Christofer Bogaso escreveu:> Hi, > > I would like to download data from below page directly onto R. > > http://www.nseindia.com/live_market/dynaContent/live_watch/equities_stock_watch.htm > > Could you please assist me how can I do that programmatically. > > Thanks for your time. > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >
FWIW. This violates their terms of service, unless you have their permission: http://www.nseindia.com/global/content/termsofuse.htm <quote> You may not conduct any systematic or automated data collection activities (including scraping, data mining, data extraction and data harvesting) on or in relation to our website without our express written consent. <quote/> On Tue, Aug 25, 2015 at 1:10 PM, Christofer Bogaso < bogaso.christofer at gmail.com> wrote:> Hi, > > I would like to download data from below page directly onto R. > > > http://www.nseindia.com/live_market/dynaContent/live_watch/equities_stock_watch.htm > > Could you please assist me how can I do that programmatically. > > Thanks for your time. > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >-- Schrodinger's backup: The condition of any backup is unknown until a restore is attempted. Yoda of Borg, we are. Futile, resistance is, yes. Assimilated, you will be. He's about as useful as a wax frying pan. 10 to the 12th power microphones = 1 Megaphone Maranatha! <>< John McKown [[alternative HTML version deleted]]