This question will probably get me in trouble on theoretical grounds, but I will pose it anyway. The situation: I recently ran a field study looking for differences in sugarbeet cultivar tolerance to a specific herbicide. The study was set up so that 37 cultivars were treated with 4 different applications of the herbicide (37*4 factorial). In doing so, we found that the interaction effect was highly insignificant (ndf=108, ddf=144, F=0.28, p=1.0000). Now my problem is this... the study takes up an enormous amount of time, energy, and money (as you could guess with 37 cultivars in a field study). We need to determine weather it is worth the effort to repeat the study this summer (practically, it is not, but our funding source would like a more concrete demonstration). I decided to try using power.anova.test just as a starting point to see what our power was. My question is: is this valid to do on an interaction term? If I use power.anova.test with on the interaction term, this is what I get: ~> power.anova.test(groups=(37*4), n=3, between.var=12.06, ~+ within.var=21.23, sig.level=0.05) ~ ~ Balanced one-way analysis of variance power calculation ~ ~ groups = 148 ~ n = 3 ~ between.var = 12.06 ~ within.var = 21.23 ~ sig.level = 0.05 ~ power = 1 ~ ~ NOTE: n is number in each group This would imply that given the variability we observed with 3 replications, we almost certainly would have found differences if they existed. But given what I have read on power analysis, a high p-value and wide confidence intervals nearly always suggest inadequate sample size. (Our 90% confidence intervals differed from the estimates by as much as 28%, when a 10% difference would be significant from a practical perspective.) So is this a valid approach? Or does the power.anova.test fall apart if using an interaction effect? Thank you in advance for any help or references you are willing to point me to. Best regards, Andrew Kniss Assistant Research Scientist University of Wyoming Department of Plant Sciences 1000 E. University Ave. Laramie, WY 82071 USA akniss at uwyo.edu
Your F value is so low as to make me suspect your model. Where did the 144 denominator degrees of freedom come from? Andrew Kniss wrote:> This question will probably get me in trouble on theoretical grounds, but I > will pose it anyway. > > The situation: > I recently ran a field study looking for differences in sugarbeet cultivar > tolerance to a specific herbicide. The study was set up so that 37 > cultivars were treated with 4 different applications of the herbicide (37*4 > factorial). In doing so, we found that the interaction effect was highly > insignificant (ndf=108, ddf=144, F=0.28, p=1.0000). Now my problem is > this... the study takes up an enormous amount of time, energy, and money (as > you could guess with 37 cultivars in a field study). We need to determine > weather it is worth the effort to repeat the study this summer (practically, > it is not, but our funding source would like a more concrete demonstration). > > I decided to try using power.anova.test just as a starting point to see what > our power was. My question is: is this valid to do on an interaction term? > If I use power.anova.test with on the interaction term, this is what I get: > > ~> power.anova.test(groups=(37*4), n=3, between.var=12.06, > ~+ within.var=21.23, sig.level=0.05) > ~ > ~ Balanced one-way analysis of variance power calculation > ~ > ~ groups = 148 > ~ n = 3 > ~ between.var = 12.06 > ~ within.var = 21.23 > ~ sig.level = 0.05 > ~ power = 1 > ~ > ~ NOTE: n is number in each group > > > This would imply that given the variability we observed with 3 replications, > we almost certainly would have found differences if they existed. But given > what I have read on power analysis, a high p-value and wide confidence > intervals nearly always suggest inadequate sample size. (Our 90% confidence > intervals differed from the estimates by as much as 28%, when a 10% > difference would be significant from a practical perspective.) > > So is this a valid approach? Or does the power.anova.test fall apart if > using an interaction effect? > > Thank you in advance for any help or references you are willing to point me > to. > Best regards, > Andrew Kniss > Assistant Research Scientist > University of Wyoming > Department of Plant Sciences > 1000 E. University Ave. > Laramie, WY 82071 USA > > akniss at uwyo.edu > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html >-- Bob Wheeler --- http://www.bobwheeler.com/ ECHIP, Inc. --- Randomness comes in bunches.
Using the data set posted at http://uwstudentweb.uwyo.edu/A/AKNISS/sbxherb.txt I obtained this output from R:> data(sbxherb) > attach(sbxherb) > sbxherb$rep<-factor(c("I","II","III")) > sbxherb$var<-factor(sbxherb$var) > sbxherb$trt<-factor(sbxherb$trt, labels=c("Check","Early","Late","Micro")) > detach(sbxherb) > attach(sbxherb) > aov.sbx<-aov(yield~rep + trt * var + Error(rep/(trt+var))) > summary(aov.sbx)Error: rep Df Sum Sq Mean Sq rep 2 122.158 61.079 Error: rep:trt Df Sum Sq Mean Sq F value Pr(>F) trt 3 201.131 67.044 1.3096 0.355 Residuals 6 307.160 51.193 Error: rep:var Df Sum Sq Mean Sq F value Pr(>F) var 36 4613.9 128.2 8.4392 1.425e-14 *** Residuals 72 1093.5 15.2 --- Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1 Error: Within Df Sum Sq Mean Sq F value Pr(>F) trt:var 108 956.62 8.86 0.9731 0.5574 Residuals 216 1966.04 9.10 I would like to use this output to help design next years study, which will be carried out with the same design. However, I would like to know what the sample size should be in order to have an 80% chance of finding a 10% difference in the interaction term trt:var at alpha=0.05. Is there a way to coerce power.anova.test() to make this calculation for me? I have tried using this function to give me the sample size needed to find differences given the within and between groups variance from the previous study, but with no success. If I have it calculate the power from this data set, it tells me I have a power of 1, which I doubt is true given the high p-value and wide confidence intervals around the estimates.> power.anova.test(groups=148,+ sig.level=0.05, + power=0.8, + within.var=9.10, + between.var=8.86) Error in uniroot(function(n) eval(p.body) - power, c(2, 1e+05)) : f() values at end points not of opposite sign> power.anova.test(groups=148,+ sig.level=0.05, + n=3, + within.var=9.10, + between.var=8.86) Balanced one-way analysis of variance power calculation groups = 148 n = 3 between.var = 8.86 within.var = 9.1 sig.level = 0.05 power = 1 NOTE: n is number in each group Any help is greatly appreciated. I apologize if I seem to be nagging on this issue, but I thought this post was much better at describing what I am after. Thanks in advance. Andrew Kniss University of Wyoming akniss at uwyo.edu -----Original Message----- From: Andrew Criswell [mailto:andrew at arcriswell.com] Sent: Tuesday, February 22, 2005 9:30 PM To: akniss at uwyo.edu Subject: Re: [R] power.anova.test for interaction effects Dear Andrew, If you could supply us with your data, more help would possibly be forthcoming. Best wishes, Andrew Andrew Criswell, PhD Graduate School, Bangkok University