From a quick look at the paper in the SAS proceedings, the simulations
seem limited to nested designs. The major problems are with repeated
measures designs where the error structure is not compound symmetric,
which lme4 does not at present handle (unless I have missed something).
Such imbalance as was investigated was not a serious issue, at least for
the Kenward and Roger degree of freedom calculations.
The paper ends by commenting that "research should continue". What
may be even more important is to educate users to think carefully about
any df that they are presented with, and to be especially sceptical when
designs are not approximately balanced nested designs and/or there are
repeated measures error structures that are not compound symmetric.
It is also necessary to consider how well the analysis reflects matters
on which there may be existing good evidence. Suppose in Ronaldo's
case that he'd previously run a number of experiments with very similar
plots and observation al units, and with comparable treatments and
outcome measures. If the plot 1 SD estimate (i.e., at the level of
experimental units) had never been larger than 0.01, with the SD for
observational units always in a range of 2 to 20, I'd take this as
licence
to ignore the variance at the plot 1 level. It would be nice to be
able to
build in such prior information more formally, probably via a modified
version of mcmcsamp().
[Some people are never satisfied, You've written a great piece of
software, and users reward you by complaining that they want even
more!]
John Maindonald.
On 1 Jan 2006, at 10:00 PM, r-help-request at stat.math.ethz.ch wrote:
> From: Dave Atkins <datkins at u.washington.edu>
> Date: 1 January 2006 1:40:45 AM
> To: r-help at stat.math.ethz.ch
> Subject: Re: [R] lme X lmer results
>
>
>
> Message: 18
> Date: Fri, 30 Dec 2005 12:51:59 -0600
> From: Douglas Bates <dmbates at gmail.com>
> Subject: Re: [R] lme X lmer results
> To: John Maindonald <john.maindonald at anu.edu.au>
> Cc: r-help at stat.math.ethz.ch
> Message-ID:
> <40e66e0b0512301051i2dc0f257r745c70e749c250f0 at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> On 12/29/05, John Maindonald <john.maindonald at anu.edu.au> wrote:
>
> >> Surely there is a correct denominator degrees of freedom if the
> design
> >> is balanced, as Ronaldo's design seems to be. Assuming that he
has
> >> specified the design correctly to lme() and that lme() is
> getting the df
> >> right, the difference is between 2 df and 878 df. If the t-
> statistic
> >> for the
> >> second level of Xvar had been 3.0 rather than 1.1, the difference
> >> would be between a t-statistic equal to 0.095 and 1e-6. In a
> design
> >> where there are 10 observations on each experimental unit, and all
> >> comparisons are at the level of experimental units or above, df
for
> >> all comparisons will be inflated by a factor of at least 9.
>
> Doug Bates commented:
>
> I don't want to be obtuse and argumentative but I still am not
> convinced that there is a correct denominator degrees of freedom for
> _this_ F statistic. I may be wrong about this but I think you are
> referring to an F statistic based on a denominator from a different
> error stratum, which is not what is being quoted. (Those are not
> given because they don't generalize to unbalanced designs.)
>
> This is why I would like to see someone undertake a simulation study
> to compare various approaches to inference for the fixed effects terms
> in a mixed model, using realistic (i.e. unbalanced) examples.
>
> Doug--
>
> Here is a paper that focused on the various alternatives to
> denominator degrees of freedom in SAS and does report some
> simulation results:
>
> http://www2.sas.com/proceedings/sugi26/p262-26.pdf
>
> Not sure whether it argues convincingly one way or the other in the
> present discussion.
>
> cheers, Dave
>
> --
> Dave Atkins, PhD
> datkins at u.washington.edu
>
