Hi,
I have a data set that consists of about 2 million observations and several
high dimensional fixed effects (2 factors at around 1000 levels each, and
others with a few hundred levels). I'm looking to run linear and logit
regressions. I've tried packages such as filehash and biglm to store some of
the big matrices on the hard drive, but I still get errors like "Cannot
allocate vector of length ...".
I've read about some iterative methods for coefficient estimation in STATA
that would probably work for this, but I'm wondering if there is an R
package out there meant for situations like mine. I'm running a XP x64
machine with an AMD 2.8Ghz dual core processor and 6GB of RAM, and I'm not
really concerned with memory- and time-intensive solutions as long as they
work.
Thanks,
Dan
--
View this message in context:
http://www.nabble.com/Large-data-sets-with-high-dimensional-fixed-effects-tp25894824p25894824.html
Sent from the R help mailing list archive at Nabble.com.
