Hi, I am trying to fi a glm-poisson model to 400.000 records. I have tried biglm and glmulti but i have problems... can it really be the case that 400.000 are too many records??? I am thinking of using random samples of my dataset..... Many thanks, -- View this message in context: http://r.789695.n4.nabble.com/glm-poisson-fitting-400-000-records-tp3925100p3925100.html Sent from the R help mailing list archive at Nabble.com.
D_Tomas <tomasmeca <at> hotmail.com> writes:> > Hi, > > I am trying to fi a glm-poisson model to 400.000 records. I have tried biglm > and glmulti but i have problems... can it really be the case that 400.000 > are too many records??? > > I am thinking of using random samples of my dataset..... >"I have problems" isn't enough for us to diagnose. I tried this trivial example in base R:> d <- data.frame(x=runif(4e5),y=rpois(4e5,5)) > system.time(glm(y~x,family=poisson,data=d,trace=TRUE))Deviance = 438614.6 Iterations - 1 Deviance = 417968.2 Iterations - 2 Deviance = 417921.2 Iterations - 3 Deviance = 417921.2 Iterations - 4 user system elapsed 5.444 12.952 18.429 Can you give us a hint about what went wrong??
My apologies for my vague comment. My data comprises 400.000 x 21 (17 explanatory variables, plus response variable, plus two offsets). If I build the full model (only linear) I get: Error: cannot allocate vector of size 112.3 Mb I have a 4GB RAM laptop... Would i get any improvemnt on a 8GB computer Many thanks, -- View this message in context: http://r.789695.n4.nabble.com/glm-poisson-fitting-400-000-records-tp3925100p3925968.html Sent from the R help mailing list archive at Nabble.com.