Dear helpeRs, a colleague of mine would like to give R a try. He uses econometric models which typically involve a large number of variables, esp. time series. Having no experience with handling very large data sets myself I turn to you. 1. Could you please describe your experiences to cope with these situations? 2. What kind of difficulties will he have to face? Are there special tricks (packages) he might try? 3. Can you recommend to use R? Sorry, if my question is a bit vague but at this point I'm not able to give any further details. Any help is very much appreciated. D. Trenkler -- Dietrich Trenkler c/o Universitaet Osnabrueck Rolandstr. 8; D-49069 Osnabrueck, Germany email: Dietrich.Trenkler at Uni-Osnabrueck.de
Dear Dietrich, in the first place, it would have been helpful to know which kind of econometric models your colleague wants to utilise. With respect to econometric methods you might want to have a look at the CRAN Task Views for econometrics and finance, to see what is already available: http://cran.at.r-project.org/src/contrib/Views/Econometrics.html http://cran.at.r-project.org/src/contrib/Views/Finance.html>From a practical point of view, I have by now not encountered any size limitations; these experiences are gained from working with high frequency multivariate data in the finance context as well as simultaneous multiple equation models that are solved on a per period basis by applying the Gau?-Seidel algorithm; my colleagues are using R in the context of stock selection where on a daily basis multivariate time series data of more than 3,000 strocks are processed. We have not encountered any problems so far. Basically, the maximal data handling size is determined by your computer's memory and the style of your coding.Best, Bernhard> >Dear helpeRs, > >a colleague of mine would like to give R a try. He uses econometric >models which typically involve a large number of variables, esp. time >series. Having no experience with handling very large data sets myself >I turn to you. > >1. Could you please describe your experiences to cope with these > situations? > >2. What kind of difficulties will he have to face? Are there special > tricks (packages) he might try? > >3. Can you recommend to use R? > > >Sorry, if my question is a bit vague but at this point I'm not able to >give any further details. > >Any help is very much appreciated. > >D. Trenkler > >-- >Dietrich Trenkler c/o Universitaet Osnabrueck >Rolandstr. 8; D-49069 Osnabrueck, Germany >email: Dietrich.Trenkler at Uni-Osnabrueck.de > >______________________________________________ >R-help at r-project.org mailing list >https://stat.ethz.ch/mailman/listinfo/r-help >PLEASE do read the posting guide >http://www.R-project.org/posting-guide.html >and provide commented, minimal, self-contained, reproducible code. >***************************************************************** Confidentiality Note: The information contained in this ...{{dropped:10}}
One approach econometricians use with linear models is to demean the data in order to reduce the computational burden. This can be done using the ave() function. But, this is still difficult because, in my experience, you still need to build a model matrix that must be demeaned and that model matrix may be big. On the other hand, if you're dealing with large sparse data sets, you may find either the sparseM and Matrix packages useful as they are designed to handle large sparse matrices.> -----Original Message----- > From: r-help-bounces at r-project.org > [mailto:r-help-bounces at r-project.org] On Behalf Of Dietrich Trenkler > Sent: Wednesday, November 07, 2007 3:05 AM > To: R-help > Subject: [R] Using R for large econometric models > > Dear helpeRs, > > a colleague of mine would like to give R a try. He uses > econometric models which typically involve a large number of > variables, esp. time series. Having no experience with > handling very large data sets myself I turn to you. > > 1. Could you please describe your experiences to cope with these > situations? > > 2. What kind of difficulties will he have to face? Are there special > tricks (packages) he might try? > > 3. Can you recommend to use R? > > > Sorry, if my question is a bit vague but at this point I'm > not able to give any further details. > > Any help is very much appreciated. > > D. Trenkler > > -- > Dietrich Trenkler c/o Universitaet Osnabrueck > Rolandstr. 8; D-49069 Osnabrueck, Germany > email: Dietrich.Trenkler at Uni-Osnabrueck.de > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >
I take it your friend is interested in large scale macro econometric models involving perhaps more than a thousand equations. The software one would like to use might involve managing the model database, estimating the model and simulating the model. If the model involves rational expectations these simulations will be very demanding. R would probably handle a lot of the estimation requirements and probably most of the data management. I do not know of anyone that has handled the more elaborate RE simulations in R. It would involve considerable programming. The only free software that does this kind of work is the Fair-Parke program available from http://fairmodel.econ.yale.edu/. If the model has been set up by another person you might be advised to use the software that the maintainers of the model use. TROLL is widely used in Central Banks and organisations such as the IMF ECB etc. It would be a fairly big job to translate a large scale macro model from a system such as TROLL or MODELEASY or similar to another language. If the model is relatively small then any statistical system that can store data, estimate structural equations and simulate them is ideal. R would be suitable in this case. If RE are involved you may need special software or some programming. Best Regards John On 07/11/2007, Dietrich Trenkler <Dietrich.Trenkler at uni-osnabrueck.de> wrote:> Dear helpeRs, > > a colleague of mine would like to give R a try. He uses econometric > models which typically involve a large number of variables, esp. time > series. Having no experience with handling very large data sets myself > I turn to you. > > 1. Could you please describe your experiences to cope with these > situations? > > 2. What kind of difficulties will he have to face? Are there special > tricks (packages) he might try? > > 3. Can you recommend to use R? > > > Sorry, if my question is a bit vague but at this point I'm not able to > give any further details. > > Any help is very much appreciated. > > D. Trenkler > > -- > Dietrich Trenkler c/o Universitaet Osnabrueck > Rolandstr. 8; D-49069 Osnabrueck, Germany > email: Dietrich.Trenkler at Uni-Osnabrueck.de > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >-- John C Frain Trinity College Dublin Dublin 2 Ireland www.tcd.ie/Economics/staff/frainj/home.html mailto:frainj at tcd.ie mailto:frainj at gmail.com