R help - Sep 2015 - Error from lme4: "Error: (p <- ncol(X)) == ncol(Y) is not TRUE"

Hello all,?I am trying to run a random intercept model using lme4. The random
effect is a factor of 29 possibilities, making a model with one random effect
(one level). It is just a linear model. There are 713 observations. However,
when trying to run the model I receive the error?"Error: (p <- ncol(X))
== ncol(Y) is not TRUE",
a search for which reveals somewhat surprisingly little. Has anyone seen this
before? Note that if I simply change the random effect into a fixed effect and
use lm, the model works perfectly.Thank you!Rory
  
	[[alternative HTML version deleted]]

Rory,

When I searched online, I found an issue with lme4 on GitHub that suggests
this error is "due to NA values in non-factor variables".
https://github.com/lme4/lme4/issues/246

Hope this helps.

Jean

On Tue, Sep 22, 2015 at 8:18 AM, Rory Wilson <roryrwilson at yahoo.ca>
wrote:
> Hello all, I am trying to run a random intercept model using lme4. The
> random effect is a factor of 29 possibilities, making a model with one
> random effect (one level). It is just a linear model. There are 713
> observations. However, when trying to run the model I receive the
> error "Error: (p <- ncol(X)) == ncol(Y) is not TRUE",
> a search for which reveals somewhat surprisingly little. Has anyone seen
> this before? Note that if I simply change the random effect into a fixed
> effect and use lm, the model works perfectly.Thank you!Rory
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
	[[alternative HTML version deleted]]

On 23/09/15 01:18, Rory Wilson wrote:
> Hello all, I am trying to run a random intercept model using lme4.
> The random effect is a factor of 29 possibilities, making a model
> with one random effect (one level). It is just a linear model. There
> are 713 observations. However, when trying to run the model I receive
> the error "Error: (p <- ncol(X)) == ncol(Y) is not TRUE", a
search
> for which reveals somewhat surprisingly little. Has anyone seen this
> before? Note that if I simply change the random effect into a fixed
> effect and use lm, the model works perfectly.Thank you!
[Caveat:  I really find the syntax of lmer() incomprehensible, so my 
example below could be a load of dingos' kidneys.]

I think a reproducible example (as specified by the posting guide) is 
needed here.  When I do:

set.seed(42)
f <- factor(sample(1:29,713,TRUE))
x <- seq(0,1,length=713)
y <- rnorm(713)
require(lme4)
fit <- lmer(y ~ x + (1|f))

I get a reasonable (???) looking result and no error messages.

cheers,

Rolf Turner

-- 
Technical Editor ANZJS
Department of Statistics
University of Auckland
Phone: +64-9-373-7599 ext. 88276

Hi Rolf,Yes, a reprodicuble example would be good - but it appears to be trouble
with something within the data, and I cannot upload all the data here. Jean
indicated the issue could be with NAs in the data, and I am inclined to agree. I
will look more into that angle and see if something becomes apparent.Thanks for
your help!Rory
      From: Rolf Turner <r.turner at auckland.ac.nz>
 To: Rory Wilson <roryrwilson at yahoo.ca>; "r-help at
r-project.org" <r-help at r-project.org>
 Sent: Tuesday, September 22, 2015 11:26 PM
 Subject: Re: [FORGED] [R] Error from lme4: "Error: (p <- ncol(X)) ==
ncol(Y) is not TRUE"
   



On 23/09/15 01:18, Rory Wilson wrote:
> Hello all, I am trying to run a random intercept model using lme4.
> The random effect is a factor of 29 possibilities, making a model
> with one random effect (one level). It is just a linear model. There
> are 713 observations. However, when trying to run the model I receive
> the error "Error: (p <- ncol(X)) == ncol(Y) is not TRUE", a
search
> for which reveals somewhat surprisingly little. Has anyone seen this
> before? Note that if I simply change the random effect into a fixed
> effect and use lm, the model works perfectly.Thank you!
[Caveat:? I really find the syntax of lmer() incomprehensible, so my 
example below could be a load of dingos' kidneys.]

I think a reproducible example (as specified by the posting guide) is 
needed here.? When I do:

set.seed(42)
f <- factor(sample(1:29,713,TRUE))
x <- seq(0,1,length=713)
y <- rnorm(713)
require(lme4)
fit <- lmer(y ~ x + (1|f))

I get a reasonable (???) looking result and no error messages.

cheers,

Rolf Turner


  
	[[alternative HTML version deleted]]

In reply to Rolf Turner and Jean Adams who have been helping me:
This does appear to be an issue with NA values in the non-factor variables. In
the (non-reproducible) example below, we can see that removing the NAs solves
the problem. However, from what I can see to this point, there does not seem be
be rhyme nor reason to why the issue is taking place. A slight modification to
Rolf Turner's code (introducing some NAs) shows that, in general, NAs are
not a problem for lmer (indeed, it just runs na.omit as default).
Examining which factors are affected by the removal of the NAs shows no
discernible pattern - no factors disappeared, none became "1" or
anything of this nature.I will be able to proceed just by performing the na.omit
beforehand, but it is curious.?
Thanks for your help everyone (especially Rolf Turner and Jean Adams)!

mod1<-lmer(beta~expData+techVar$RIN+techVar$sample_storage_time+(1|techVar$p_amplification))#Error:
(p <- ncol(X)) == ncol(Y) is not TRUE
mod1<-lm(beta~expData+techVar$RIN+techVar$sample_storage_time+techVar$p_amplification)#No
error given.
? #trial with eliminating NAs? elimSamps<-which(is.na(beta))?
length(elimSamps) #[1] 4
? #eliminate the NAs from all vectors? betaVals<-beta[-elimSamps]?
expD<-expData[-elimSamps])? techRIN<-techVar$RIN[-elimSamps]?
techTime<-techVar$sample_storage_time[-elimSamps]?
techPlate<-factor(techVar$p_amplification[-elimSamps])

mod1<-lmer(betaVals~expD+techRIN+techTime+(1|techPlate))
summary(mod1)#Linear mixed model fit by REML ['lmerMod']#Formula:
betaVals ~ expD + techRIN + techTime + (1 | techPlate)##REML criterion at
convergence: -1701.7##Scaled residuals:# ? ?Min ? ? ?1Q ?Median ? ? ?3Q ? ?
Max#-2.3582 -0.6996 -0.1085 ?0.6079 ?4.6743##Random effects:# Groups ? ?Name ? ?
? ?Variance ?Std.Dev.# techPlate (Intercept) 1.645e-05 0.004056# Residual ? ? ?
? ? ? ?4.991e-03 0.070644#Number of obs: 709, groups: ?techPlate, 29##Fixed
effects:# ? ? ? ? ? ? Estimate Std. Error t value#(Intercept) 2.159e-01
?9.871e-02 ? 2.188#expD ? ? ? ?2.330e-03 ?1.498e-02 ? 0.156#techRIN ? ?
5.096e-03 ?4.185e-03 ? 1.218#techTime ? ?1.399e-06 ?1.565e-05 ?
0.089##Correlation of Fixed Effects:# ? ? ? ? (Intr) expD ? tchRIN#expD ? ?
-0.919#techRIN ?-0.272 -0.103#techTime -0.206 ?0.063 ?0.021

summary(techPlate)# plate01 ?plate02 ?plate03 ?plate04 ?plate05 ?plate06
?plate07 ?plate09# ? ? ? 1 ? ? ? 22 ? ? ? 34 ? ? ? 28 ? ? ? 31 ? ? ? 28 ? ? ? 32
? ? ? 10#plate09a ?plate10 ?plate11 ?plate13 ?plate14 ?plate15 ?plate16
?plate17# ? ? ?16 ? ? ? 17 ? ? ? 15 ? ? ? ?4 ? ? ? 52 ? ? ? 55 ? ? ? 41 ? ? ?
33# plate18 ?plate19 ?plate20 ?plate21 ?plate22 ?plate23 ?plate24 ?plate25# ? ?
?50 ? ? ? 42 ? ? ? 21 ? ? ? 50 ? ? ? 13 ? ? ? 22 ? ? ? 17 ? ? ? ?7# plate26
?plate27 ?plate28 ?plate30 ?plate32# ? ? ?25 ? ? ? 21 ? ? ? ?5 ? ? ? 13 ? ? ? ?4
summary(techVar$p_amplification)# plate01 ?plate02 ?plate03 ?plate04 ?plate05
?plate06 ?plate07 ?plate09# ? ? ? 1 ? ? ? 22 ? ? ? 34 ? ? ? 28 ? ? ? 31 ? ? ? 28
? ? ? 32 ? ? ? 10#plate09a ?plate10 ?plate11 ?plate13 ?plate14 ?plate15 ?plate16
?plate17# ? ? ?17 ? ? ? 17 ? ? ? 15 ? ? ? ?4 ? ? ? 53 ? ? ? 55 ? ? ? 41 ? ? ?
33# plate18 ?plate19 ?plate20 ?plate21 ?plate22 ?plate23 ?plate24 ?plate25# ? ?
?50 ? ? ? 42 ? ? ? 21 ? ? ? 50 ? ? ? 13 ? ? ? 22 ? ? ? 17 ? ? ? ?8# plate26
?plate27 ?plate28 ?plate30 ?plate32# ? ? ?25 ? ? ? 21 ? ? ? ?5 ? ? ? 14 ? ? ? ?4

#Counter-example, where it functions fine#Example from Rolf Turner
set.seed(42)f <- factor(sample(1:29,713,TRUE))x <- seq(0,1,length=713)y
<- rnorm(713)require(lme4)
x[sample(1:713,4,replace=F)]<-NA
fit <- lmer(y ~ x + (1|f))#No error message given
  
	[[alternative HTML version deleted]]