Dear list, Sorry for posting a borderline statistical question on the list, but hte SPSS people around me just stares at me blankly when refering to tests with any term other than ANOVA and post-hoc. I would appreciate any insight on how this all is possible: I have a model fitted by aov() stored in "ppdur", which gives this result when using ANOVA:> anova(ppdur)Analysis of Variance Table Response: PAPositionPercentOfVoweldur Df Sum Sq Mean Sq F value Pr(>F) UtteranceType 4 24731 6183 2.7642 0.02696 * SyllLable 1 14584 14584 6.5202 0.01094 * Cycle 1 798 798 0.3566 0.55067 Speaker 2 9975 4987 2.2297 0.10855 Label 1 2008 2008 0.8979 0.34377 UtteranceType:SyllLable 4 15210 3803 1.7001 0.14854 UtteranceType:Cycle 4 13192 3298 1.4745 0.20855 SyllLable:Cycle 1 11306 11306 5.0545 0.02497 * UtteranceType:Speaker 7 13721 1960 0.8764 0.52488 SyllLable:Speaker 2 1291 645 0.2885 0.74951 Cycle:Speaker 2 10753 5377 2.4038 0.09135 . UtteranceType:Label 4 3579 895 0.4000 0.80871 SyllLable:Label 1 4499 4499 2.0114 0.15670 Cycle:Label 1 229 229 0.1022 0.74929 Speaker:Label 2 1241 620 0.2774 0.75788 UtteranceType:SyllLable:Cycle 3 473 158 0.0705 0.97571 UtteranceType:SyllLable:Speaker 6 13919 2320 1.0372 0.40006 UtteranceType:Cycle:Speaker 3 1221 407 0.1820 0.90865 SyllLable:Cycle:Speaker 2 1457 729 0.3258 0.72210 UtteranceType:SyllLable:Label 2 3823 1911 0.8545 0.42607 UtteranceType:Cycle:Label 3 8566 2855 1.2766 0.28160 SyllLable:Cycle:Label 1 3575 3575 1.5983 0.20669 UtteranceType:Speaker:Label 4 2658 664 0.2970 0.87990 SyllLable:Speaker:Label 2 139 70 0.0311 0.96938 Cycle:Speaker:Label 2 13599 6800 3.0400 0.04866 * UtteranceType:SyllLable:Cycle:Speaker 2 2015 1008 0.4505 0.63757 UtteranceType:SyllLable:Cycle:Label 1 11 11 0.0051 0.94328 UtteranceType:SyllLable:Speaker:Label 1 603 603 0.2695 0.60386 Residuals 539 1205605 2237 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Ok now, when I want to know where the differences are, I get this result from TukeyHSD:> TukeyHSD(ppdur,c("Cycle:Speaker:Label" ),ordered=TRUE)Tukey multiple comparisons of means 95% family-wise confidence level factor levels have been ordered Fit: aov(formula = PAPositionPercentOfVoweldur ~ UtteranceType * SyllLable * Cycle * Speaker * Label, data =PABTSub) $`Cycle:Speaker:Label` diff lwr upr p adj 3:Andrea:!H*L-1:Lavinia:!H*L 2.6069140 -37.499300 42.71313 1.0000000 1:Vito:!H*L-1:Lavinia:!H*L 8.7764075 -85.090794 102.64361 1.0000000 3:Andrea:H*L-1:Lavinia:!H*L 12.3411960 -18.883688 43.56608 0.9792814 1:Vito:H*L-1:Lavinia:!H*L 15.0416018 -32.746962 62.83017 0.9968844 1:Andrea:H*L-1:Lavinia:!H*L 15.2934976 -17.987036 48.57403 0.9379977 1:Lavinia:H*L-1:Lavinia:!H*L 16.9297832 -14.670124 48.52969 0.8394874 3:Lavinia:H*L-1:Lavinia:!H*L 18.3218965 -13.445765 50.08956 0.7631935 3:Lavinia:!H*L-1:Lavinia:!H*L 20.9338365 -19.932636 61.80031 0.8761167 3:Vito:!H*L-1:Lavinia:!H*L 24.3874104 -18.890036 67.66486 0.7894161 3:Vito:H*L-1:Lavinia:!H*L 27.8865684 -6.758302 62.53144 0.2589397 1:Andrea:!H*L-1:Lavinia:!H*L 28.8093072 -18.979256 76.59787 0.7077134 1:Vito:!H*L-3:Andrea:!H*L 6.1694934 -87.982875 100.32186 1.0000000 3:Andrea:H*L-3:Andrea:!H*L 9.7342820 -22.337674 41.80624 0.9977466 1:Vito:H*L-3:Andrea:!H*L 12.4346877 -35.911602 60.78098 0.9995138 1:Andrea:H*L-3:Andrea:!H*L 12.6865836 -21.389960 46.76313 0.9870844 1:Lavinia:H*L-3:Andrea:!H*L 14.3228692 -18.114318 46.76006 0.9529437 3:Lavinia:H*L-3:Andrea:!H*L 15.7149825 -16.885651 48.31562 0.9149770 3:Lavinia:!H*L-3:Andrea:!H*L 18.3269225 -23.190369 59.84421 0.9530401 3:Vito:!H*L-3:Andrea:!H*L 21.7804964 -22.112036 65.67303 0.8978922 3:Vito:H*L-3:Andrea:!H*L 25.2796544 -10.130570 60.68988 0.4470040 1:Andrea:!H*L-3:Andrea:!H*L 26.2023932 -22.143897 74.54868 0.8288881 3:Andrea:H*L-1:Vito:!H*L 3.5647885 -87.160916 94.29049 1.0000000 1:Vito:H*L-1:Vito:!H*L 6.2651943 -91.407252 103.93764 1.0000000 1:Andrea:H*L-1:Vito:!H*L 6.5170902 -84.936471 97.97065 1.0000000 1:Lavinia:H*L-1:Vito:!H*L 8.1533757 -82.702082 99.00883 1.0000000 3:Lavinia:H*L-1:Vito:!H*L 9.5454891 -81.368450 100.45943 1.0000000 3:Lavinia:!H*L-1:Vito:!H*L 12.1574291 -82.321291 106.63615 0.9999996 3:Vito:!H*L-1:Vito:!H*L 15.6110030 -79.935308 111.15731 0.9999948 3:Vito:H*L-1:Vito:!H*L 19.1101609 -72.848673 111.06899 0.9999396 1:Andrea:!H*L-1:Vito:!H*L 20.0328997 -77.639547 117.70535 0.9999471 1:Vito:H*L-3:Andrea:H*L 2.7004057 -38.577297 43.97811 1.0000000 1:Andrea:H*L-3:Andrea:H*L 2.9523016 -20.019330 25.92393 0.9999996 1:Lavinia:H*L-3:Andrea:H*L 4.5885872 -15.872500 25.04967 0.9998713 3:Lavinia:H*L-3:Andrea:H*L 5.9807005 -14.738525 26.69993 0.9985708 3:Lavinia:!H*L-3:Andrea:H*L 8.5926405 -24.425090 41.61037 0.9994563 3:Vito:!H*L-3:Andrea:H*L 12.0462144 -23.912642 48.00507 0.9946379 3:Vito:H*L-3:Andrea:H*L 15.5453724 -9.361836 40.45258 0.6595505 1:Andrea:!H*L-3:Andrea:H*L 16.4681112 -24.809592 57.74581 0.9777282 1:Andrea:H*L-1:Vito:H*L 0.2518959 -42.601917 43.10571 1.0000000 1:Lavinia:H*L-1:Vito:H*L 1.8881814 -39.673935 43.45030 1.0000000 3:Lavinia:H*L-1:Vito:H*L 3.2802948 -38.409509 44.97010 1.0000000 3:Lavinia:!H*L-1:Vito:H*L 5.8922348 -43.086576 54.87105 0.9999998 3:Vito:!H*L-1:Vito:H*L 9.3458087 -41.661963 60.35358 0.9999831 3:Vito:H*L-1:Vito:H*L 12.8449666 -31.076810 56.76674 0.9983895 1:Andrea:!H*L-1:Vito:H*L 13.7677055 -41.119471 68.65488 0.9996173 1:Lavinia:H*L-1:Andrea:H*L 1.6362855 -21.842569 25.11514 1.0000000 3:Lavinia:H*L-1:Andrea:H*L 3.0283989 -20.675753 26.73255 0.9999996 3:Lavinia:!H*L-1:Andrea:H*L 5.6403389 -29.327804 40.60848 0.9999955 3:Vito:!H*L-1:Andrea:H*L 9.0939128 -28.663734 46.85156 0.9997414 3:Vito:H*L-1:Andrea:H*L 12.5930707 -14.847220 40.03336 0.9385504 1:Andrea:!H*L-1:Andrea:H*L 13.5158096 -29.338003 56.36962 0.9968280 3:Lavinia:H*L-1:Lavinia:H*L 1.3921134 -19.888090 22.67232 1.0000000 3:Lavinia:!H*L-1:Lavinia:H*L 4.0040534 -29.368559 37.37667 0.9999998 3:Vito:!H*L-1:Lavinia:H*L 7.4576273 -28.827357 43.74261 0.9999460 3:Vito:H*L-1:Lavinia:H*L 10.9567852 -14.418986 36.33256 0.9598798 1:Andrea:!H*L-1:Lavinia:H*L 11.8795240 -29.682592 53.44164 0.9986943 3:Lavinia:!H*L-3:Lavinia:H*L 2.6119400 -30.919560 36.14344 1.0000000 3:Vito:!H*L-3:Lavinia:H*L 6.0655139 -30.365659 42.49669 0.9999937 3:Vito:H*L-3:Lavinia:H*L 9.5646718 -16.019698 35.14904 0.9866445 1:Andrea:!H*L-3:Lavinia:H*L 10.4874107 -31.202393 52.17721 0.9996066 3:Vito:!H*L-3:Lavinia:!H*L 3.4535739 -41.134704 48.04185 1.0000000 3:Vito:H*L-3:Lavinia:!H*L 6.9527318 -29.316321 43.22179 0.9999733 1:Andrea:!H*L-3:Lavinia:!H*L 7.8754707 -41.103340 56.85428 0.9999956 3:Vito:H*L-3:Vito:!H*L 3.4991580 -35.466379 42.46469 1.0000000 1:Andrea:!H*L-3:Vito:!H*L 4.4218968 -46.585875 55.42967 1.0000000 1:Andrea:!H*L-3:Vito:H*L 0.9227388 -42.999038 44.84452 1.0000000 As you can see, I don't get a significant p-value for this interaction effect anymore. How could that be? (For the other variables showing a significant effect TykeyHSD gives me information about where the effect may come from, so I did not include them in my example. Also, maybe I should point out that the names in the example are coded ones. They are NOT the acctual names of hte participants.). I would be happy to get any insight into how this could come about. /Fredrik -- "Life is like a trumpet - if you don't put anything into it, you don't get anything out of it." [[alternative HTML version deleted]]
On 21/03/2009, at 12:50 AM, Fredrik Karlsson wrote:> Dear list, > > Sorry for posting a borderline statistical question on the list, > but hte > SPSS people around me just stares at me blankly when refering to > tests with > any term other than ANOVA and post-hoc. I would appreciate any > insight on > how this all is possible: > > I have a model fitted by aov() stored in "ppdur", which gives this > result > when using ANOVA:<snip>> As you can see, I don't get a significant p-value for this interaction > effect anymore. How could that be?How? Well it just could. That's the way with statistics. Remember you're not talking about things being definitely true, you're talking about there being ``significant'' evidence that they're true. This can lead to apparent paradoxes. A simple example of such ``paradoxes'' arises in the context of multiple comparisons. In a one-way anova these might show you that level A is ``the same as'' level B, and level B is ``the same as'' level C, but nevertheless level A is *different from* level C. This is just saying that we have *evidence* that level A is different from level C. Ergo it follows, as doth the night follow the day, that level B must differ either from level A or from level C, or both. There just isn't enough information in the data to decide which of the possibilities is true. A larger data set would be able to ``make the decision''. Your example of multiple comparison results ``contradicting'' the anova results is not an unheard of phenomenon. I like to illustrate what's going on via the diagram shown in the attached pdf file. Think of the null hypothesis of ``no difference between the levels'' being rejected whenever the sample falls outside of a certain enclosure. In the diagram the circle represents the enclosure corresponding to the anova test; the square represents the enclosure corresponding to the multiple comparisons test. If the sample lands outside both the circle and the square, then both tests reject the null. But it can happen, rarely but not too rarely, that the sample lands inside one of the bits of the circle that stick out beyond the square. In this case the multiple comparisons test will say that there are differences, but the anova test will say there are none. Alternatively, the sample could land in one of the corners of the square that stick out beyond the circle. In this case the anova test will say that there are differences, but the multiple comparisons test will find none. That's just All Part of the Rich Tapestry of Life when you do statistical hypothesis testing. BTW don't take the circle and the square too literally. They are just illustrative analogies; don't try to interpret them in terms of what's really going on in the actual hypothesis testing mechanism. HTH. cheers, Rolf Turner ###################################################################### Attention: This e-mail message is privileged and confidential. If you are not the intended recipient please delete the message and notify the sender. Any views or opinions presented are solely those of the author. This e-mail has been scanned and cleared by MailMarshal www.marshalsoftware.com ###################################################################### -------------- next part -------------- A non-text attachment was scrubbed... Name: diagram.pdf Type: application/pdf Size: 4376 bytes Desc: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20090321/b89eb582/attachment-0003.pdf>