Dear List, I downloaded R for the first time yesterday, in the hopes that I might deal more effectively with a complex repeated measures experimental design involving inbred strains of laboratory mice. The design below, somewhat simplified, cannot be computed with standard ANOVA, because something called the X'X matrix is too large. The design has the following factors: Between-subject factors (levels): inbred mouse strain (20, "twenty") sex (2) Animals: 10 per sex*strain combination (400 total) Within-subject factors: drug (3) trial set (4) stimulus characteristic 1 (2) stimulus characteristic 2 (2) My question for the R community is, does R have in nlme or lme the ability to compute this problem on a typical desktop PC? In Stata, for instance, which has a matrix size limit of 11,000, the problem above will not fit, using standard ANOVA. Matrix size can be determined by listing all the terms of the full model and multiplying the levels of each factor withing a term and summing all terms. I said simplified in the first paragraph above, because if I include the day of drug challenge, the model oversteps the number of within-subject factors allowed. So, surprisingly to me, Stata/SE, which I bought for large data sets, is too small! Not that I don't like Stata, but I am annoyed that I must find another tool to use. I understand that SAS Proc Mixed will compute the problem, because it may handle the covariance matrix in some kind of piecemeal fashion (perhaps by animal but I've no confirmation of this, except that it zips through a comparable data set on someone else's computer). However, I am running Apple OS X and don't have SAS on my machine. I don't really understand what's going on underneath these programs respective hoods, but my question here is whether R computes mixed models in such a way as to require a matrix size like Stata or like SAS, when mixed models of this size are presented? Sincerely, -Dave
The short answer is "yes". The longer answer is the lme function in the nlme package uses a compact representation of the model matrices for a mixed-effects model. It is a sparse matrix representation that is specific to cases of strictly nested random effects when there is more that one level of random effects. In your case it seems that there will only be random effects for animal and the sparse matrix representation would apply. You may find that the version of lme in the experimental lme4 package is faster than the version in nlme. David Airey <david.airey at vanderbilt.edu> writes:> I downloaded R for the first time yesterday, in the hopes that I might > deal more effectively with a complex repeated measures experimental > design involving inbred strains of laboratory mice. The design below, > somewhat simplified, cannot be computed with standard ANOVA, because > something called the X'X matrix is too large. The design has the > following factors: > > > Between-subject factors (levels): > inbred mouse strain (20, "twenty") > sex (2) > Animals: > 10 per sex*strain combination (400 total) > Within-subject factors: > drug (3) > trial set (4) > stimulus characteristic 1 (2) > stimulus characteristic 2 (2) > > My question for the R community is, does R have in nlme or lme the > ability to compute this problem on a typical desktop PC? In Stata, for > instance, which has a matrix size limit of 11,000, the problem above > will not fit, using standard ANOVA. Matrix size can be determined by > listing all the terms of the full model and multiplying the levels of > each factor withing a term and summing all terms. I said simplified in > the first paragraph above, because if I include the day of drug > challenge, the model oversteps the number of within-subject factors > allowed. So, surprisingly to me, Stata/SE, which I bought for large > data sets, is too small! Not that I don't like Stata, but I am annoyed > that I must find another tool to use. I understand that SAS Proc Mixed > will compute the problem, because it may handle the covariance matrix > in some kind of piecemeal fashion (perhaps by animal but I've no > confirmation of this, except that it zips through a comparable data > set on someone else's computer). However, I am running Apple OS X and > don't have SAS on my machine. I don't really understand what's going > on underneath these programs respective hoods, but my question here is > whether R computes mixed models in such a way as to require a matrix > size like Stata or like SAS, when mixed models of this size are > presented? > > > Sincerely, > > -Dave > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://www.stat.math.ethz.ch/mailman/listinfo/r-help-- Douglas Bates bates at stat.wisc.edu Statistics Department 608/262-2598 University of Wisconsin - Madison http://www.stat.wisc.edu/~bates/