R help - Mar 2011 - ok to use glht() when interaction is NOT significant?

Hi, let's say I have a simple ANOVA model with 2 factors A (level A1 and A2)
and
B (level B1 and B2) and their interaction:

aov(y~A*B, data=dat)

It turns out that the interaction term is not significant (e.g. P value = 0.2), 
but if I used glht() to compare A1 vs. A2 within each level of B, I found that 
the comparison is not significant when B=B1, but is very significant (P<0.01)
when B=B2.

My question is whether it's legal to do this post-hoc comparison when the 
interaction is NOT significant? Can I still make the claim that there is a 
significant difference between A1 and A2 when B=B2?

Thanks

John



      
	[[alternative HTML version deleted]]

Inline below

On Mon, Mar 7, 2011 at 8:08 PM, array chip <arrayprofile at yahoo.com>
wrote:
> Hi, let's say I have a simple ANOVA model with 2 factors A (level A1
and A2) and
> B (level B1 and B2) and their interaction:
>
> aov(y~A*B, data=dat)
>
> It turns out that the interaction term is not significant (e.g. P value =
0.2),
> but if I used glht() to compare A1 vs. A2 within each level of B, I found
that
> the comparison is not significant when B=B1, but is very significant
(P<0.01)
> when B=B2.
>
> My question is whether it's legal to do this post-hoc comparison when
the
> interaction is NOT significant? Can I still make the claim that there is a
> significant difference between A1 and A2 when B=B2?
(I am serious here). Don't know what "legal" means. Why do you
want to
make the claim? When does it **ever** mean anything scientifically
meaningful to make it? What is the **scientific** question of
interest? Are the data unbalanced? Have you plotted the data to tell
you what's going on?

Warning: I come from the school (maybe I'm the only student...) that
believes all such formal post hoc comparisons are pointless, silly,
wastes of effort.  Note the word: "formal" -- that is pretending the P
values mean anything, For exploratory purposes, which can certainly
include producing P values as well as graphs, such post hoc
comparisons might lead to great science. It's the "formal" part
that I
reject and that you seem to be hung up on.

Note also: If you're a Bayesian and can put priors on everything, you
can spit out posteriors and Bayes factors to your heart's content.
Really! -- no need to sweat multiplicity even. Of course, I speak here
only as an observer, having never actually inhaled myself.*

Cheers,
Bert

*Apologies to all non-US and younger readers. This is a smart-aleck
reference to an infamous dumb remark from a recent famous, smart
former U.S. president. Google "never inhaled" for details.
>
> Thanks
>
> John
>
>
>
>
> ? ? ? ?[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>


-- 
Bert Gunter
Genentech Nonclinical Biostatistics
467-7374
http://devo.gene.com/groups/devo/depts/ncb/home.shtml