R help - May 2008 - marginality principle / selecting the right type of SS for an interaction hypothesis

Hello,

I have a problem with selecting the right type of sums of squares for
an ANCOVA for my specific experimental data and hypotheses. I do have
a basic understanding of the differences between Type-I, II, and III
SSs, have read about the principle of marginality, and read Venable's
"Exegeses on Linear Models" 
(http://www.stats.ox.ac.uk/pub/MASS3/Exegeses.pdf). I am pretty new to R 
and a search of the R-help archive did not
answer my question (although I found some good pointers).

In brief, leaving my covariates aside, I hypothesize that women (a)
generally perform lower then men in a specific task (microworld
performance, MWP) and that they (b) perform especially poor if a
certain situational condition exists ("stereotype threat"). N = 160,
80 female & 80 male participants, 82 under stereotype threat and 78 not.

I realize that it makes no sense to report/interpret a main effect of
stereotype threat in the confirmed presence of the interaction effect
GENDER:STTHREAT, because a main effect of stereotype threat would
actually be caused by the interaction (an error-bar plot illustrating
this can be found here if one scrolls a little downwars: 
http://myowelt.blogspot.com/2008/05/obtaining-same-anova-results-in-r-as-in.html)

. I thus tend to use Type-II SSs and calculate my ANOVA with
> library(car)
> Anova(lm(MWP ~ GENDER * STTHREAT), type="II")
Anova Table (Type II tests)

Response: MWP
                  Sum Sq  Df F value    Pr(>F)
GENDER           23.939   1 32.3672 6.139e-08 ***
STTHREAT         12.684   1 17.1489 5.644e-05 ***
GENDER:STTHREAT   4.997   1  6.7557   0.01024 *
Residuals       115.380 156
---
Signif. codes:  0 ?***? 0.001 ?**? 0.01 ?*? 0.05 ?.? 0.1 ? ? 1

However, it would make sense to report the main effect of GENDER in
the presence of the interaction and thus violate the marginality
principle, because of hypothesis (a) above. Would that mean that Type-
III SSs are desirable for the analysis of the main effect of GENDER,
and Type-II SSs are desirable for the main effect of STTHREAT and the
interaction? Or would it be better to specify a model that only
includes the interaction term and the main effect of gender with Type-
III SSs? Like this:
> options(contrasts=c("contr.sum", "contr.poly"))
> fit <- aov(MWP ~ GENDER:STTHREAT + GENDER)
> drop1(fit,~.,test="F")
Single term deletions

Model:
MWP ~ GENDER:STTHREAT + GENDER
                 Df Sum of Sq     RSS     AIC F value     Pr(F)
<none>                       115.380 -44.310
GENDER           1    23.381 138.761 -16.787  31.612 8.475e-08 ***
GENDER:STTHREAT  2    17.680 133.061 -25.499  11.952 1.481e-05 ***
---
Signif. codes:  0 ?***? 0.001 ?**? 0.01 ?*? 0.05 ?.? 0.1 ? ? 1

If this has been answered before and I was just too blind to find it,
I apologize and would appreciate a link to the post.

Another question that arises from the use of Type-III SSs: If every
factor is corrected for the other factors, the SSs of all factors plus
the RSS do not sum up to the total SS of the model. But doesn't that
lead to a situation where the standard way of calculating eta-square
for a factor by dividing its SS by the total SS cannot be applied?

Regards,
Bertolt

-- 
Bertolt Meyer
Senior Assistant
Psychological Institute, University of Zurich
Social Psychology
Binzmuehlestr. 14, Box 15
CH-8050 Zurich
Switzerland

bmeyer at sozpsy.uzh.ch
tel:   +41446357282
fax:   +41446357279

Dear Bertolt,

Since you have specific hypotheses in mind, why not just test them directly
as contrasts? That is, test for (1) the difference between men and women in
the absence of stereotype threat; (2) the difference between men and women
in the presence of stereotype threat; and (3) the difference in these
differences (i.e., the interaction).

The fact that you expect a positive difference in (1) and (2) doesn't make
the test for the gender "main effect" sensible, because you're
averaging
simple effects of different magnitude. 

BTW, although I didn't read it carefully, the your web-page description of a
main effect in the presence of an interaction appears to imply that this
averages across all individuals, when in fact it averages across the levels
of the other factor; the result is different when there are unequal numbers
of observations in the cells. As well, you describe the model in your email
as an ANCOVA, when in fact it appears to be a two-way ANOVA.

I hope this helps,
 John

------------------------------
John Fox, Professor
Department of Sociology
McMaster University
Hamilton, Ontario, Canada
web: socserv.mcmaster.ca/jfox
> -----Original Message-----
> From: r-help-bounces at r-project.org [mailto:r-help-bounces at
r-project.org]
On
> Behalf Of Bertolt Meyer
> Sent: May-25-08 10:51 AM
> To: r-help at r-project.org
> Subject: [R] marginality principle / selecting the right type of SS for an
> interaction hypothesis
> 
> Hello,
> 
> I have a problem with selecting the right type of sums of squares for
> an ANCOVA for my specific experimental data and hypotheses. I do have
> a basic understanding of the differences between Type-I, II, and III
> SSs, have read about the principle of marginality, and read Venable's
> "Exegeses on Linear Models"
> (http://www.stats.ox.ac.uk/pub/MASS3/Exegeses.pdf). I am pretty new to R
> and a search of the R-help archive did not
> answer my question (although I found some good pointers).
> 
> In brief, leaving my covariates aside, I hypothesize that women (a)
> generally perform lower then men in a specific task (microworld
> performance, MWP) and that they (b) perform especially poor if a
> certain situational condition exists ("stereotype threat"). N =
160,
> 80 female & 80 male participants, 82 under stereotype threat and 78
not.
> 
> I realize that it makes no sense to report/interpret a main effect of
> stereotype threat in the confirmed presence of the interaction effect
> GENDER:STTHREAT, because a main effect of stereotype threat would
> actually be caused by the interaction (an error-bar plot illustrating
> this can be found here if one scrolls a little downwars:
> http://myowelt.blogspot.com/2008/05/obtaining-same-anova-results-in-r-as-
> in.html)
> 
> . I thus tend to use Type-II SSs and calculate my ANOVA with
> 
> > library(car)
> > Anova(lm(MWP ~ GENDER * STTHREAT), type="II")
> Anova Table (Type II tests)
> 
> Response: MWP
>                   Sum Sq  Df F value    Pr(>F)
> GENDER           23.939   1 32.3672 6.139e-08 ***
> STTHREAT         12.684   1 17.1489 5.644e-05 ***
> GENDER:STTHREAT   4.997   1  6.7557   0.01024 *
> Residuals       115.380 156
> ---
> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05
'.' 0.1 ' ' 1
> 
> However, it would make sense to report the main effect of GENDER in
> the presence of the interaction and thus violate the marginality
> principle, because of hypothesis (a) above. Would that mean that Type-
> III SSs are desirable for the analysis of the main effect of GENDER,
> and Type-II SSs are desirable for the main effect of STTHREAT and the
> interaction? Or would it be better to specify a model that only
> includes the interaction term and the main effect of gender with Type-
> III SSs? Like this:
> 
> > options(contrasts=c("contr.sum", "contr.poly"))
> > fit <- aov(MWP ~ GENDER:STTHREAT + GENDER)
> > drop1(fit,~.,test="F")
> Single term deletions
> 
> Model:
> MWP ~ GENDER:STTHREAT + GENDER
>                  Df Sum of Sq     RSS     AIC F value     Pr(F)
> <none>                       115.380 -44.310
> GENDER           1    23.381 138.761 -16.787  31.612 8.475e-08 ***
> GENDER:STTHREAT  2    17.680 133.061 -25.499  11.952 1.481e-05 ***
> ---
> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05
'.' 0.1 ' ' 1
> 
> If this has been answered before and I was just too blind to find it,
> I apologize and would appreciate a link to the post.
> 
> Another question that arises from the use of Type-III SSs: If every
> factor is corrected for the other factors, the SSs of all factors plus
> the RSS do not sum up to the total SS of the model. But doesn't that
> lead to a situation where the standard way of calculating eta-square
> for a factor by dividing its SS by the total SS cannot be applied?
> 
> Regards,
> Bertolt
> 
> --
> Bertolt Meyer
> Senior Assistant
> Psychological Institute, University of Zurich
> Social Psychology
> Binzmuehlestr. 14, Box 15
> CH-8050 Zurich
> Switzerland
> 
> bmeyer at sozpsy.uzh.ch
> tel:   +41446357282
> fax:   +41446357279
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.