On Sun, 10 Oct 2004, RenE J.V. Bertin wrote:> Could I ask some hints/help in interpreting the following ANOVA results, > please? This concerns an experiment where I study the incidence and > severity of motion sickness. I have Sickness.norm, a subjective > discomfort/sickness estimate, normalised to 0..1, the session time T > (normalised to 0..1 and binned in 0.2 wide bins) and a qualitive > indicator if a given Subject was sick or not. For instance, to see if > there is an effect of time, I do > > > > summary( aov.SS1( Sickness.norm~WasSick*T.norm.Class + Error(Subject/(WasSick*T.norm.Class)), na.action=na.exclude ) ) > Factor "WasSick", levels: 0 1 > Factor "T.norm.Class", levels: 0 0.2 0.4 0.6 0.8 1 > Sun Oct 10 13:45:45 2004 > WasSick data: 377 > Warning in aov(Sickness.norm ~ WasSick * T.norm.Class + Error(Subject/(WasSick*T.norm.Class)), na.action=naa, : > Error model is singular > > Error: Subject > Df Sum Sq Mean Sq F value Pr(>F) > WasSick 1 11.170 11.170 21.037 0.0001 *** > T.norm.Class 5 5.058 1.012 1.905 0.1278 > WasSick:T.norm.Class 5 1.680 0.336 0.633 0.6765 > Residuals 26 13.805 0.531 > --- > Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1 > > Error: Subject:T.norm.Class > Df Sum Sq Mean Sq F value Pr(>F) > T.norm.Class 5 10.916 2.183 13.673 3.29e-11 *** > WasSick:T.norm.Class 5 2.318 0.464 2.903 0.0153 * > Residuals 169 26.984 0.160 > > > > There is a very nice effect of time (the line T.norm.Class under > Subject:T.norm.Class). There is no interaction between the two factors > (that would have been given on a line WasSick:T.norm.Class under the > Subject:WasSick:T.norm.Class caption, right?!).Not right.> That means that the > effect of time is not different for the two groups, which is true: > sickness increases for both, only less so for the non-sick group. The > time effect is significant only for the sick group, when tested > seperately. But what I do not manage to grasp is the interpretation of > the other significant effects:> *1 What does the very significant WasSick effect (under the Error: > Subject caption) mean? That there is a sign. difference between the > subjects who were sick and who were not, which is indeed very true?See next point.> *2 What is the meaning of the interaction WasSick:T.norm.Class > (p=0.0153)? Can I interpret this as meaning that sickness increases > significantly less in the non-sick group?Here you have a problem, as that term appears in two strata, so presumably the nesting was unbalanced. You either need to use Yates' `recovery of inter-block information' or, simpler, use lme. You have evidence that the effect of WasSick differs by T.norm.Class. If that is substantiated by a more refined analysis, it does not make sense to say there is a `very significant WasSick effect', as it is likely there is an effect for some session times and not others, or at least that the size of the effect differs by session time. Trying to interpret main effects in the presence of interactions depends on the (unstated) coding of factors used. If this is treatment coding, WasSick is the effect at T=0 (significant) and the effect appears to change with time.> *3 What importance should I give to the warning that the error model is > singular? Does this stem from the fact that Subject and WasSick are not > independent? R 1.8 generally refused to calculate this sort of test: can > I rely on the results R 1.9.1 gives me??I think that means the correct error model is Error(Subject/T.norm.Class): my guess is that WasSick is a subject-level observation and so each subject only has one level of it. Certainly that is the model which was fitted. -- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595
On Sun, 10 Oct 2004, RenE J.V. Bertin wrote:> On Sun, 10 Oct 2004 18:04:23 +0100 (BST), Prof Brian Ripley <ripley at stats.ox.ac.uk> wrote regarding > "Re: [R] some help interpreting ANOVA results, please?" > > Thank you for your answers. > > 8-) Here you have a problem, as that term appears in two strata, so presumably > 8-) the nesting was unbalanced. You either need to use Yates' `recovery of > 8-) inter-block information' or, simpler, use lme. > > Would you have any suggestions for an accessible text explaining lme, apart from MASS-4? > > 8-) You have evidence that the effect of WasSick differs by T.norm.Class. > 8-) If that is substantiated by a more refined analysis, it does not make > 8-) sense to say there is a `very significant WasSick effect', as it is likely > 8-) there is an effect for some session times and not others, or at least that > 8-) the size of the effect differs by session time. > > WasSick is an overall assessment made at the end of the experimental sessions. That is why I tend to use it as an independent variable. Of course I did not decide whether or not a given subject would be sick (WasSick==1) or not, nor was it designated by a random process. It is in a way intrinsically linked to the individual subjects, not unlike gender. Maybe that is something to take into account for choosing the right test? > > 8-) Trying to interpret main effects in the presence of interactions depends > 8-) on the (unstated) coding of factors used. If this is treatment coding, > 8-) WasSick is the effect at T=0 (significant) and the effect appears to > 8-) change with time. > > I am not sure what you mean with (treatment) coding: aoptions(contrasts=c("contr.treatment", something)) uses treatment coding for unordered factors. That's the default in R, but not usually what is used in ANOVA.> between-subjects design where some subjects receive a particular > treatment and other don't? For WasSick, this comes down to the same > thing as my design, and also for T.norm.Class, in a certain way. I have > a number of subjects who all did a nauseogenic task (driving a car > simulator), and indicated their level of discomfort while doing so. > There was no strict timing protocol, and besides, each subject will have > his/her own time scale on which the sickness eveolves and/or is > indicated, hence my use of normalised time. WasSick at T=0 is not of > interest, as it is 0, by definition. > > > 8-) I think that means the correct error model is Error(Subject/T.norm.Class): > 8-) my guess is that WasSick is a subject-level observation and so each > 8-) subject only has one level of it. Certainly that is the model which was > 8-) fitted. > > Yes, that is true (see above), and there are not as many sick as non-sick subjects (although the difference is not huge). When I use the Error term as you suggest, I do indeed get the same results, but also still the warning message. > > Fortunately, when I run the test on only one group, the results are much clearer (N.S. in the non-sick): > > > with2( SelectCases(ss600.3, "WasSick==1"), summary( aov( Sickness.norm~T.norm.Class + Error(Subject/(T.norm.Class)) ) ) ) > Warning in aov(Sickness.norm ~ T.norm.Class + Error(Subject/(T.norm.Class))) : > Error model is singular > > Error: Subject > Df Sum Sq Mean Sq F value Pr(>F) > T.norm.Class 5 3.716 0.743 2.191 0.110 > Residuals 15 5.087 0.339 > > Error: Subject:T.norm.Class > Df Sum Sq Mean Sq F value Pr(>F) > T.norm.Class 5 11.561 2.312 16.15 2.25e-11 *** > Residuals 91 13.027 0.143 > > RenE Bertin > >-- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595
RenE J.V. Bertin wrote:>... > > Would you have any suggestions for an accessible text explaining lme, apart from MASS-4? >The standard reference on lme is Pinheiro and Bates (2000) Mixed-Effects Models in S and S-Plus (Springer). I have found this book very valuable (as I have Bates' other book on nonlinear regression with Don Watts). hope this helps. spencer graves
Christoph Scherber
2005-Feb-16 10:35 UTC
[R] some help interpreting ANOVA results, please?
Dear RenE, Can you explain a bit more how you derive your T.SPart? That?s what I think is the tricky part of your analysis. I would suggest you should try to end up with something like this: model1<-aov(SR~WasSick*Time+Error(Subject/Time) model2<-aov(SR~SC*Time+Error(Subject/Time) This way it would be like a repeated measures ANOVA, where WasSick (or SC) are the primary covariates, and Time is nested within Subject. I think the correct specification of "time" is crucial for the whole analysis. It?s like in a split-plot ANOVA, where finding the appropriate codings for plots of different sizes can sometimes take a very long time. Regards, Christoph 0) Subject, the subject identifier 1) physiological recordings, say SR (skin resistance): time series 2) a SessionPart variable (parts R1 and R2, separated in time by a pause) 3) time, T.SPart: normalised per subject and per SessionPart, so twice 0..1 4) a subjective sickness estimate (SC): time series 5) a per-subject classification: WasSick or not (available as a time series, but constant in time of course) RenE J.V. Bertin wrote:>On Sun, 10 Oct 2004 19:55:41 +0200, "RenE J.V. Bertin" <rjvbertin at hotmail.com> wrote regarding "Re: >[R] some help interpreting ANOVA results, please?" > >I'm would like to come back to a question I posted quite a while ago, concerning the analysis of data of an ongoing experiment. I have, for a given number of subjects: >0) Subject, the subject identifier >1) physiological recordings, say SR (skin resistance): time series >2) a SessionPart variable (parts R1 and R2, separated in time by a pause) >3) time, T.SPart: normalised per subject and per SessionPart, so twice 0..1 >4) a subjective sickness estimate (SC): time series >5) a per-subject classification: WasSick or not (available as a time series, but constant in time of course) > >I would like to make statements on whether or not sickness (measured by 4 or 5) can be deduced from the physiological recordings, e.g. something like > > >>aov( SR ~ WasSick * T.SPart ) >> >> > >expecting a significant effect of time (sickness building up), of WasSick, and a significant interaction showing that the effect is stronger (or only significant) in the WasSick=TRUE subjects. A simple t.test(SR~WasSick) gives a significant difference, as well as t.test( SR~ (T.SPart>=0.5) ) . > >The problem I'm having is that WasSick (and SC) are not independent variables properly speaking. So I cannot do > > > >>aov( SR ~ WasSick * T.SPart + Error(Subject/WasSick*T.SPart) ) >> >> > >R would remove WasSick from the Error term, and do the analysis without it, giving a significant T.SPart effect and WasSick:T.SPart interaction (?), both listed under Error: Subject:T.SPart : >Error: Subject:T.SPart > Df Sum Sq Mean Sq F value Pr(>F) >T.SPart 5 318.2 63.6 8.336 7.46e-07 *** >WasSick:T.SPart 5 125.5 25.1 3.289 0.0079 ** >Residuals 129 984.9 7.6 > >There is no trace of a WasSick effect other than in that interaction (of which I'm not sure it is truly one). > > > >I have 2 questions at this point: > >A) I think one could assimilate WasSick to a grouping variable (like in a clinical stdudy), forgetting it is actually an observation on the subjects. In that case, I could do > > >>aov( SR ~ WasSick * T.SPart ) >> >> >which gives me the expected two significant main effects and the significant interaction (which agrees with visual inspection of the data). >Is this an acceptable approach/model? > >B) Should I contine putting the Subject id in an Error term, e.g. > > >>aov( SR ~ WasSick + Error(Subject) ) >> >> >WithOUT this error term, that anova gives a significant effect, confirming the t.test mentioned above. If I include the error term, the effect is no longer significant. >Is that because the model does not make sense, rather because my data are so non-normal that a t.test cannot be used? (?Error has a similar model, and calls it "not particularly sensible statistically".) > > >I would really appreciate some more constructive comments! >Thanks, >RenE Bertin > >PS: I must add that it has been suggested to try lme. I went over what docs I have (help and MASS 4), but these are far to specialistic for me, so I haven't gotten anywhere in that direction :( > >______________________________________________ >R-help at stat.math.ethz.ch mailing list >https://stat.ethz.ch/mailman/listinfo/r-help >PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html > > >