R help - Feb 2006 - Strange p-level for the fixed effect with lme function

Hello,
I ran two lme analyses and got expected results. However, I saw
something suspicious regarding p-level for fixed effect. Models are the
same, only experimental designs differ and, of course, subjects. I am
aware that I could done nesting Subjects within Experiments, but it is
expected to have much slower RT (reaction time) in the second
experiment, since the task is more complex, so it would not make much
sense. That is why I kept analyses separated:

(A) lme(RT ~ F2 + MI, random =~ 1 | Subject, data = exp1)

ANOVA:
            numDF denDF   F-value p-value
(Intercept)     1  1379 243012.61  <.0001
F2              1  1379     47.55  <.0001
MI              1  1379      4.69  0.0305

Fixed effects: RT ~ F2 + MI
                Value  Std.Error   DF   t-value p-value
(Intercept)  6.430962 0.03843484 1379 167.32118  0.0000
F2          -0.028028 0.00445667 1379  -6.28896  0.0000
MI          -0.004058 0.00187358 1379  -2.16612  0.0305

==========================================================
(B) lme(RT ~ F2 + MI, random =~ 1 | Subject, data = exp2)

ANOVA:
            numDF denDF   F-value p-value
(Intercept)     1   659 150170.71  <.0001
F2              1   659     17.28  <.0001
MI              1   659     13.43   3e-04

Fixed effects: RT ~ F2 + MI
                Value  Std.Error  DF   t-value p-value
(Intercept)  6.608252 0.05100954 659 129.54935  0.0000
F2          -0.008679 0.00616191 659  -1.40855  0.1594
MI           0.009476 0.00258605 659   3.66420  0.0003

As you can see, in exp1 p-levels for the model and for the fixed effects
are the same, as thay should be, as far as I know. Yet, in exp2 there is
significant p for F2 in the model, but insignificant regarding F2 as
fixed factor. How is it possible? I have ran many linear models and
those two values correspond (or are the same). Anyway, how can it be to
have insignificant effect that is significant in the model? Some strange
property of that factor, like distribution? Multicolinearity? Please,
help me on that.

Sincerely,
Petar

Petar Milin <pmilin at ff.ns.ac.yu> writes:
> Hello,
> I ran two lme analyses and got expected results. However, I saw
> something suspicious regarding p-level for fixed effect. Models are the
> same, only experimental designs differ and, of course, subjects. I am
> aware that I could done nesting Subjects within Experiments, but it is
> expected to have much slower RT (reaction time) in the second
> experiment, since the task is more complex, so it would not make much
> sense. That is why I kept analyses separated:
> 
> (A) lme(RT ~ F2 + MI, random =~ 1 | Subject, data = exp1)
> 
> ANOVA:
>             numDF denDF   F-value p-value
> (Intercept)     1  1379 243012.61  <.0001
> F2              1  1379     47.55  <.0001
> MI              1  1379      4.69  0.0305
> 
> Fixed effects: RT ~ F2 + MI
>                 Value  Std.Error   DF   t-value p-value
> (Intercept)  6.430962 0.03843484 1379 167.32118  0.0000
> F2          -0.028028 0.00445667 1379  -6.28896  0.0000
> MI          -0.004058 0.00187358 1379  -2.16612  0.0305
> 
> ==========================================================> 
> (B) lme(RT ~ F2 + MI, random =~ 1 | Subject, data = exp2)
> 
> ANOVA:
>             numDF denDF   F-value p-value
> (Intercept)     1   659 150170.71  <.0001
> F2              1   659     17.28  <.0001
> MI              1   659     13.43   3e-04
> 
> Fixed effects: RT ~ F2 + MI
>                 Value  Std.Error  DF   t-value p-value
> (Intercept)  6.608252 0.05100954 659 129.54935  0.0000
> F2          -0.008679 0.00616191 659  -1.40855  0.1594
> MI           0.009476 0.00258605 659   3.66420  0.0003
> 
> As you can see, in exp1 p-levels for the model and for the fixed effects
> are the same, as thay should be, as far as I know. Yet, in exp2 there is
> significant p for F2 in the model, but insignificant regarding F2 as
> fixed factor. How is it possible? I have ran many linear models and
> those two values correspond (or are the same). Anyway, how can it be to
> have insignificant effect that is significant in the model? Some strange
> property of that factor, like distribution? Multicolinearity? Please,
> help me on that.
"Type I"... 

The  ANOVA is progressive, so refers to the situation *after* removing
MI from the model.  Try anova(lmefit, Terms="F2")

-- 
   O__  ---- Peter Dalgaard             ??ster Farimagsgade 5, Entr.B
  c/ /'_ --- Dept. of Biostatistics     PO Box 2099, 1014 Cph. K
 (*) \(*) -- University of Copenhagen   Denmark          Ph:  (+45) 35327918
~~~~~~~~~~ - (p.dalgaard at biostat.ku.dk)                  FAX: (+45) 35327907

What code did you actually run to get what you labelled as 'ANOVA'?

If this was anova[.lme], the default type is "sequential", whereas the
t-values (from summary[.lme], I presume) are from marginal tests.

Whether sequential and marginal tests are similar or even the same is a 
question of balance in the design (for linear models as well).

On Thu, 23 Feb 2006, Petar Milin wrote:
> Hello,
> I ran two lme analyses and got expected results. However, I saw
> something suspicious regarding p-level for fixed effect. Models are the
> same, only experimental designs differ and, of course, subjects. I am
> aware that I could done nesting Subjects within Experiments, but it is
> expected to have much slower RT (reaction time) in the second
> experiment, since the task is more complex, so it would not make much
> sense. That is why I kept analyses separated:
>
> (A) lme(RT ~ F2 + MI, random =~ 1 | Subject, data = exp1)
>
> ANOVA:
>            numDF denDF   F-value p-value
> (Intercept)     1  1379 243012.61  <.0001
> F2              1  1379     47.55  <.0001
> MI              1  1379      4.69  0.0305
>
> Fixed effects: RT ~ F2 + MI
>                Value  Std.Error   DF   t-value p-value
> (Intercept)  6.430962 0.03843484 1379 167.32118  0.0000
> F2          -0.028028 0.00445667 1379  -6.28896  0.0000
> MI          -0.004058 0.00187358 1379  -2.16612  0.0305
>
> ==========================================================>
> (B) lme(RT ~ F2 + MI, random =~ 1 | Subject, data = exp2)
>
> ANOVA:
>            numDF denDF   F-value p-value
> (Intercept)     1   659 150170.71  <.0001
> F2              1   659     17.28  <.0001
> MI              1   659     13.43   3e-04
>
> Fixed effects: RT ~ F2 + MI
>                Value  Std.Error  DF   t-value p-value
> (Intercept)  6.608252 0.05100954 659 129.54935  0.0000
> F2          -0.008679 0.00616191 659  -1.40855  0.1594
> MI           0.009476 0.00258605 659   3.66420  0.0003
>
> As you can see, in exp1 p-levels for the model and for the fixed effects
> are the same, as thay should be, as far as I know. Yet, in exp2 there is
> significant p for F2 in the model, but insignificant regarding F2 as
> fixed factor. How is it possible? I have ran many linear models and
> those two values correspond (or are the same). Anyway, how can it be to
> have insignificant effect that is significant in the model? Some strange
> property of that factor, like distribution? Multicolinearity? Please,
> help me on that.
-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595