R help - Nov 2008 - Randomization of a two-way ANOVA?

Hello list,

I wish to perform a randomization test on the F-statistics of a 2 way ANOVA
but have not been able to find out how to do so - is there a package  /
function that can perform this that I am unaware of? 

FactorA has 6 levels (0,1,2,3,4,5) whereas FactorB has 3 (1,2,3). A sample:

Resp.	FactorA	FactorB
2	0	2
3	1	2
1	2	2
0	3	2
0	4	2
0	5	2
4	0	1
6	1	1
1	2	1
0	3	1
1	4	1
0	5	1
2	0	2
3	1	2
1	2	2
2	3	2
1	4	2
0	5	2
3	0	1
3	1	1
1	2	1
0	3	1
0	4	1
7	1	3
2	2	3
0	3	3
1	4	3
0	5	3
1	0	3

Also, is the F-statistic an appropriate test-statistic for the randomization
test when it is computed using non-normal data?

Any insight into this issue would be much appreciated.
Thanks.

-- 
View this message in context:
http://www.nabble.com/Randomization-of-a-two-way-ANOVA--tp20761769p20761769.html
Sent from the R help mailing list archive at Nabble.com.

The code below may not be the most optimized, but it should do the trick.
I've implemented the permutation test, which by some nomenclatures is
distinguished from the randomization test in that the latter is exhaustive
while the former is not.

I'm eager to hear the views of the list on whether the permutation test
fares when you use a parametric stat like the F as the focal statistic. I
know that David Howell introduction to the test employs the F (
http://www.uvm.edu/~dhowell/StatPages/Resampling/Resampling.html). I
personally think that in cases of known violation of normality and/or
homogeneity of variance, it seems odd to incorporate MSerror into the focal
statistic. I look forward to the list educating me if this view turns out to
be uninformed.

#make sure the factors are treated as such
FactorA = factor(FactorA)
FactorB = factor(FactorB)

#get the observed Fs
obs.aov = aov(Resp~FactorA*FactorB)
obs.mainA.F = summary(obs.aov)[[1]]$F[1]
obs.mainB.F = summary(obs.aov)[[1]]$F[2]
obs.AbyB.F = summary(obs.aov)[[1]]$F[3]

#prepare for the permutation loop
perms = 1e4
perm.mainA.F = rep(NA,perms)
perm.mainB.F = rep(NA,perms)
perm.AbyB.F = rep(NA,perms)

#run the permutation loop
for(i in 1:perms){
    Randomized.Resp = Resp[order(runif(length(Resp)))]
    perm.aov = aov(Randomized.Resp~FactorA*FactorB)
    perm.mainA.F[i] = summary(perm.aov)[[1]]$F[1]
    perm.mainB.F[i] = summary(perm.aov)[[1]]$F[2]
    perm.AbyB.F[i] = summary(perm.aov)[[1]]$F[3]
}

#compute the p-values
p.mainA = mean(perm.mainA.F>obs.mainA.F)
p.mainB = mean(perm.mainB.F>obs.mainB.F)
p.AbyB = mean(perm.AbyB.F>obs.AbyB.F)

#print the p-values
print(p.mainA)
print(p.mainB)
print(p.AbyB)




-- 
Mike Lawrence
Graduate Student
Department of Psychology
Dalhousie University
www.thatmike.com

Looking to arrange a meeting? Do so at:
http://www.timetomeet.info/with/mike/

~ Certainty is folly... I think. ~

	[[alternative HTML version deleted]]

This really depends on the question(s) that you are asking.

If you want a simultaneous test of all your factors, then you can randomly
permute the response many times and see where the original f-stat (or others)
falls in the distribution of the randomized stats.

If you want to test the interaction, then fit the full model and save the f-stat
for the interaction term.  Fit the model without the interaction and save the
residuals and fitted values, permute the residuals and add them back to the
fitted values, then analyze this with the full model and save the f-stat. 
Repeate and see how the f-stat (or other stat) of the original fit compares to
the permuted stats.

For combinations of other questions, just do the same as above but with the full
and reduced model that matches the question(s) of interest.

You could also do bootstrapping as a randomization test, there are a couple of
packages that help with that (but it is also easy without the packages), but for
testing, I prefer the permutation test.

Hope this helps,

--
Gregory (Greg) L. Snow Ph.D.
Statistical Data Center
Intermountain Healthcare
greg.snow at imail.org
801.408.8111

> -----Original Message-----
> From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-
> project.org] On Behalf Of Joe Ratster
> Sent: Sunday, November 30, 2008 1:22 PM
> To: r-help at r-project.org
> Subject: [R] Randomization of a two-way ANOVA?
>
>
> Hello list,
>
> I wish to perform a randomization test on the F-statistics of a 2 way
> ANOVA
> but have not been able to find out how to do so - is there a package  /
> function that can perform this that I am unaware of?
>
> FactorA has 6 levels (0,1,2,3,4,5) whereas FactorB has 3 (1,2,3). A
> sample:
>
> Resp.   FactorA FactorB
> 2       0       2
> 3       1       2
> 1       2       2
> 0       3       2
> 0       4       2
> 0       5       2
> 4       0       1
> 6       1       1
> 1       2       1
> 0       3       1
> 1       4       1
> 0       5       1
> 2       0       2
> 3       1       2
> 1       2       2
> 2       3       2
> 1       4       2
> 0       5       2
> 3       0       1
> 3       1       1
> 1       2       1
> 0       3       1
> 0       4       1
> 7       1       3
> 2       2       3
> 0       3       3
> 1       4       3
> 0       5       3
> 1       0       3
>
> Also, is the F-statistic an appropriate test-statistic for the
> randomization
> test when it is computed using non-normal data?
>
> Any insight into this issue would be much appreciated.
> Thanks.
>
> --
> View this message in context: http://www.nabble.com/Randomization-of-a-
> two-way-ANOVA--tp20761769p20761769.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.