Why are the results not reliable?
________________________________
From: ESCHEN Rene [mailto:rene.eschen@unifr.ch]
Sent: Wednesday, August 23, 2006 3:48 AM
To: Spencer Graves; r-help@stat.math.ethz.ch
Cc: Doran, Harold
Subject: RE: [R] Random structure of nested design in lme
The output of the suggested lmer model looks very similar to the output of aov,
also when I ran the model on the dataset I want to use. Thank you very much for
the suggestion, this appears to solve my problem to a great extend.
However, one of my response variables is survival of my plants, which is a
binary variable (alive = 1; dead = 0). To analyze this case, I added family =
"binomial" to the command line:
fit.lme4 <-
lmer(binary.response~soiltype*habitat+(1|destination)+(1|origin), Dat0, family =
"binomial")
> anova(fit.lme4)
Analysis of Variance Table
Df Sum Sq Mean Sq Denom F value Pr(>F)
soiltype 1 0.029 0.029 32.000 0.0238 0.8784
habitat 1 0.029 0.029 32.000 0.0238 0.8784
soiltype:habitat 1 0.062 0.062 32.000 0.0504 0.8237
It seems to me that the results are either suspiciously signficant (P <
0.0001) or the other way aroud (all P > 0.75). I read in previous posts that
I am not the first to encounter this problem, but I did not find a way around
this so far.
Below I added the data I used as an additional column in the sample dataset I
used before.
Does anyone have a suggestion how to get reliable output from lmer models if
the response variable is binary?
René.
Additional column for sample dataset:
___
binary.response
0
0
0
1
0
1
0
1
1
0
0
1
0
0
0
1
0
0
1
0
1
0
1
1
0
0
0
1
0
0
0
1
1
0
0
0
___
-----Original Message-----
From: Spencer Graves [mailto:spencer.graves@pdf.com]
Sent: Fri 2006-08-04 01:35
To: ESCHEN Rene
Cc: Doran, Harold; r-help@stat.math.ethz.ch
Subject: Re: [R] Random structure of nested design in lme
I'm not familiar with 'aov', but I have two observations
that might
help you:
1. UNESTIMABLE VARIANCE COMPONENT
The variance component 'soiltype' is not estimable in your
'lme' model:
lme(NA.1~soiltype*habitat,random=~1|destination/soiltype)
That's because each level of 'soiltype' occurs only once within
each
level of 'destination' in the self-contained example you provided
below.
To confirm this, I deleted 'soiltype' from this model:
fit.lme <- lme(response~soiltype*habitat, random=~1|destination/origin)
fit.lme0 <- lme(response~soiltype*habitat, random=~1|destination)
The answers seemed to be identical except for one thing:
> VarCorr(fit.lme)
Variance StdDev
destination = pdLogChol(1)
(Intercept) 0.004149471 0.06441639
origin = pdLogChol(1)
(Intercept) 0.060968550 0.24691810
Residual 0.007265180 0.08523603
> VarCorr(fit.lme0)
destination = pdLogChol(1)
Variance StdDev
(Intercept) 0.004149471 0.06441639
Residual 0.068233730 0.26121587
The "Residual" variance in "fit.lme0" equals the
sum of "origin" and
"Residual" variances in "fit.lme".
It would help if 'lme' checked for situations like this and
either
refused to run or dropped inestimable variance components. However,
it's possible that there are so many ways that variance components can
be inestimable that it's just not feasible to check for them all. (The
function 'varcomp' in S-Plus 6.2 has the same problem.)
2. CROSSED OR NESTED?
Are 'destination' and 'origin' crossed or nested in
your 'aov' model:
aov(response~soiltype*habitat+Error(destination+origin))
I have not used 'aov', and I don't think I should take
the time now
to try to figure this out. However, this model specification suggests
to me that 'destination' and 'origin' might be crossed not
nested. (The
difference is the 'destination:origin' interaction: If
'destination+origin' is crossed, their interaction is used as the error
term; otherwise, it looks to me like you have a saturated model.) By
contrast, 'destination/origin' in lme is 'nested', which means
that the
variance component for 'origin' is in essence the crossed term and the
interaction combined.
I believe there is a way to estimate crossed random effects using
'lme', but I don't understand how. Fortunately, we can do it using
'lmer' in the 'lme4' and 'Matrix' packages.
Because of potential conflicts between 'nlme' and
'lme4', I always
quit R and restart when I switch from one to another. The following
will then fit something using 'lmer' that looks like it might match
your
'aov' fit:
library(lme4)
fit.lme4 <- lmer(
response~soiltype*habitat
+(1|destination)+(1|origin), Dat0)
where Dat0 is a data.frame with columns 'response', 'soiltype',
'habitat', 'destination' and 'origin'.
I don't know 'aov' well enough to determine easily if the
results
from this 'lmer' fit match those from 'aov', but I hope this
helps.
Spencer Graves
ESCHEN Rene wrote:
> Spencer,
>
> Thank you for the kind and elaborate reply to my previous post.
>
> I did consider the option you suggested and many variations.
Depending on the order of the random factors, lme will either
give the same output as the aov model for soiltype or for habitat,
but not both in the same model.
>
> The closest I came was
>
> anova(lme(NA.1~soiltype*habitat,random=~1|destination/soiltype))
>
> However, it apppears that in this case the interaction is tested at the
same level as soiltype.
>
> In this post, a small sample dataset with a brief explanation of the
meaning of the different column titles is included below. Also, I included both
the aov model and the lme model.
>
> Hopefully, this will help to get closer to a solution to my problem.
>
> Best regards,
>
> René Eschen.
>
> ___
>
> #Small sample dataset
> #
> data=read.table("Sample dataset.csv",header=T)
> require(nlme)
> soiltype=factor(soiltype)
> habitat=factor(habitat)
> destination=factor(destination)
> origin=factor(origin)
> summary(aov(response~soiltype*habitat+Error(destination+origin)))
> anova(lme(response~soiltype*habitat,random=~1|destination/origin))
> #
> #"habitat" type is either 'arable' or
'grassland'
> #"destination" indicates what site the soil was transplanted
into, and is considered a random factor within habitat type
> #"soiltype" is either 'arable' or 'grassland'
> #"origin" indicates what site the soil was taken from, and is
considered a random factor within soil type
> #"response" is the response variable, typically some plant
parameter such as growth rate or number of leaves, but in this example it is a
random number between 0 and 1.
> #
> "habitat" "destination" "soiltype"
"origin" "response"
> 1 1 1 1 0.63
> 1 2 1 1 0.76
> 1 3 1 1 0.14
> 2 4 1 1 0.27
> 2 5 1 1 0.88
> 2 6 1 1 0.41
> 1 1 1 2 0.47
> 1 2 1 2 0.48
> 1 3 1 2 0.76
> 2 4 1 2 0.83
> 2 5 1 2 0.88
> 2 6 1 2 0.57
> 1 1 1 3 0.80
> 1 2 1 3 0.31
> 1 3 1 3 0.22
> 2 4 1 3 0.53
> 2 5 1 3 0.97
> 2 6 1 3 0.30
> 1 1 2 4 0.46
> 1 2 2 4 0.99
> 1 3 2 4 0.56
> 2 4 2 4 0.32
> 2 5 2 4 0.46
> 2 6 2 4 0.64
> 1 1 2 5 0.03
> 1 2 2 5 0.41
> 1 3 2 5 0.24
> 2 4 2 5 0.60
> 2 5 2 5 0.04
> 2 6 2 5 0.30
> 1 1 2 6 0.97
> 1 2 2 6 0.60
> 1 3 2 6 0.22
> 2 4 2 6 0.16
> 2 5 2 6 0.58
> 2 6 2 6 0.21
>
>
>
> -----Original Message-----
> From: Spencer Graves [mailto:spencer.graves@pdf.com]
> Sent: Sat 2006-07-22 20:03
> To: ESCHEN Rene
> Cc: Doran, Harold; r-help@stat.math.ethz.ch
> Subject: Re: [R] Random structure of nested design in lme
>
> Have you considered the following:
>
> anova(lme(NA.1~soiltype*habitat,random=~1|destination/origin))
>
> This seems more closely to match the 'aov' command in your
original
> post. This model might be written in more detail as follows:
>
> NA.1[s, h, i,j,k] = b0 + ST[s] + H[h] +
> ST.H[s[i],j[j] j] + d[i] + o[i,j] + e[i,j,k]
>
> where b0 = a constant to be estimated,
>
> s = the soil type for that particular sample,
>
> h = the habitat for that sample,
>
> ST = soil type coefficients to be estimated subject to a
constraint
> that they sum to 0,
>
> H = habitat coefficients to be estimated subject to the constraint
> that they sum to 0,
>
> ST.H = soil type by habitat interaction coefficients to be
estimated
> subject to constraints that ST.H[s,.] sum to 0 and ST.H[., h] also sum
> to 0,
>
> d[i] = a random deviation associated with each destination,
assuming
> the d's are all normal, independent, with mean 0 and unknown but
> constant variance s2.d
>
> o[i, j] = a random deviation associated with each destination /
> origin combination, assuming the o's are all normal, independent, with
> mean 0 and unknown variance s2.o,
>
> and e[i,j,j] = the standard unknown noise term, normal, independent
> with mean 0 and unknown variance s2.e.
>
> The model you wrote includes nested noise terms for soil type and
> habitat as well. These terms are not estimable, which makes the answers
> garbage, but the 'lme' function does not check for replicates and
> therefore sometimes gives garbage answers without warning.
>
> To get more information from the fit, I suggest you first try
> 'methods(class="lme")', and review help pages associated
with what you
> see listed there.
>
> Have you looked at Pinheiro and Bates (2000) Mixed-Effects Models
in
> S and S-Plus (Springer)? This is my all-time favorite reference on
> Bates has been one of the leading original contributors in variance
> components analysis and nonlinear estimation more generally for over 25
> years. The 'nlme' package is the product of his work and the work
of
> many of his graduate students prior to 2000. The book, at least from my
> perspective, is very well written. Moreover, the standard R
> distribution includes files named "ch01.R", "ch02.R",
..., "ch06.R",
> "ch08.R" with the R scripts accompanying each chapter in the
book in
> "~\library\nlme\scripts" under the R installation directory on
your hard
> drive, e.g. "D:\Program files\R\R-2.3.1\library\nlme\scripts",
on my
> computer. There are minor changes in the syntax in a few places between
> the book and the current R implementation that make it impossible to get
> some of the published answers. Using these script files increases the
> likelihood that you will get essentially the book's answers and
won't be
> defeated by subtle typographical errors or by the difference between x^2
> and I(x^2), for example.
>
> If you would like further information from this listserver, please
> submit another post, preferably including a "commented, minimal,
> self-contained, reproducible code", as suggested in the posting guide
> "www.R-project.org/posting-guide.html".
>
> Hope this helps.
> Spencer Graves
>
> ESCHEN Rene wrote:
>> Although I know it's not correct, this is what I tried in lme:
>>
>>
anova(lme(NA.1~soiltype*habitat,random=~1|destination/habitat/origin/soiltype))
>>
>> # numDF denDF F-value p-value
>> #(Intercept) 1 130 12.136195 0.0007
>> #soiltype 1 130 15.099792 0.0002
>> #habitat 1 10 0.699045 0.4226
>> #soiltype:habitat 1 130 2.123408 0.1475
>>
>> René.
>>
>> -----Original Message-----
>> From: Doran, Harold [mailto:HDoran@air.org]
>> Sent: Wed 2006-07-19 13:53
>> To: ESCHEN Rene; r-help@stat.math.ethz.ch
>> Subject: RE: [R] Random structure of nested design in lme
>>
>> Can you provide an example of what you have done with lme so we might
be able to evaluate the issue?
>>
>>> -----Original Message-----
>>> From: r-help-bounces@stat.math.ethz.ch
>>> [mailto:r-help-bounces@stat.math.ethz.ch] On Behalf Of ESCHEN Rene
>>> Sent: Wednesday, July 19, 2006 7:37 AM
>>> To: r-help@stat.math.ethz.ch
>>> Subject: [R] Random structure of nested design in lme
>>>
>>> All,
>>>
>>> I'm trying to analyze the results of a reciprocal transplant
>>> experiment using lme(). While I get the error-term right in
>>> aov(), in lme() it appears impossible to get as expected. I
>>> would be greatful for any help.
>>>
>>> My experiment aimed to identify whether two fixed factors
>>> (habitat type and soil type) affect the development of
>>> plants. I took soil from six random sites each of two types
>>> (arable and grassland) and transplanted them back into the
>>> sites of origin in such way that in each of the sites there
>>> were six pots containing arable soil and six pots of
>>> grassland soil, each containing a seedling.
>>>
>>> With aov(), I got the analysis as I expected, with habitat
>>> type tested against destination site, and soil type tested
>>> against origin site:
>>>
>>> summary(aov(response~soiltype*habitat+Error(destination+origin)))
>>> #
>>> #Error: destination
>>> # Df Sum Sq Mean Sq F value Pr(>F)
>>> #habitat 1 1.0000 1.0000 0.699 0.4226
>>> #Residuals 10 14.3056 1.4306
>>> #
>>> #Error: origin
>>> # Df Sum Sq Mean Sq F value Pr(>F)
>>> #soiltype 1 1.77778 1.77778 11.636 0.006645 **
>>> #Residuals 10 1.52778 0.15278
>>> #---
>>> #Signif. codes: 0 '***' 0.001 '**' 0.01
'*' 0.05 '.' 0.1 ' ' 1 #
>>> #Error: Within
>>> # Df Sum Sq Mean Sq F value Pr(>F)
>>> #soiltype:habitat 1 0.2500 0.2500 2.1774 0.1427
>>> #Residuals 120 13.7778 0.1148
>>>
>>> However, when I try to replicate this analysis in lme, I am
>>> unable to get the structure of the random factors (origin and
>>> destination) correct. Does anyone have a suggestion how to
>>> resolve this problem?
>>>
>>> Thanks in advance.
>>>
>>> René Eschen
>>>
>>> CABI Bioscience Centre Switzerland
>>> Rue des Grillons 1
>>> 2800 Delémont
>>> Switzerland
>>>
>>> [[alternative HTML version deleted]]
>>>
>>>
>>
>> [[alternative HTML version deleted]]
>>
>>
>>
>>
------------------------------------------------------------------------
>>
>> ______________________________________________
>> R-help@stat.math.ethz.ch mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
>
> [[alternative HTML version deleted]]
>
>
>
> ------------------------------------------------------------------------
>
> ______________________________________________
> R-help@stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]
