Mark Difford
2007-Jul-21 12:58 UTC
[R] Effect size, interactions, and main effects (Stats Question) [ffmanova]
Dear List Members, I would very much appreciate any pointers you could give me on the following matter: Main Question: To what extent does the "rule" that it is unreasonable to talk about main effects if there are significant interactions in a model depend upon effect size [of the significant interaction terms]? Or is this not an issue? More practically: Suppose I were to carry out a so-called Type-II MANOVA (using ffmanova) and were to find that the interaction term in a 2-way analysis has borderline significance (say p = 0.045) and a small effect size, whereas one of the main effects is highly signficant (say p = 6.8e-10) and has a large effect size. Would it in this case be reasonable for me to ignore the interaction term, and talk only about main effects? And, presuming the main question is fair, are there general guidlines concerning the relationship between level of significance and effect size for interaction terms. Thank you in advance for your help, Mark. -- View this message in context: http://www.nabble.com/Effect-size%2C-interactions%2C-and-main-effects-%28Stats-Question%29--ffmanova--tf4121771.html#a11722075 Sent from the R help mailing list archive at Nabble.com.
Robert A LaBudde
2007-Jul-21 14:00 UTC
[R] Effect size, interactions, and main effects (Stats Question) [ffmanova]
Statistical significance is "detectability", and depends upon the size of the sample as well as the effect. A large enough experiment will result in statistical detectability of almost every interaction action term allowed. This is why estimation, not testing, has become the consensus recommendation in statistics. As a practical matter, evaluate the combined effect of your model terms with and without the interaction term(s) you are worried about. Is the reduction in accuracy of physical importance? If so, the interaction terms are required for scientific reasons. If not, present both results and indicate the acceptability (for interpolation) of the simpler model. You should also make it your first priority to hypothecate why the interaction terms are meaningful and expected. If a cause can be found, it may suggest an alternate model that will eliminate interactions, or satisfy your anxiety. If not, it may support your argument to simplify. At 08:58 AM 7/21/2007, Mark wrote:>Dear List Members, > >I would very much appreciate any pointers you could give me on the following >matter: > >Main Question: >To what extent does the "rule" that it is unreasonable to talk about main >effects if there are significant interactions in a model depend upon effect >size [of the significant interaction terms]? Or is this not an issue? > >More practically: Suppose I were to carry out a so-called Type-II MANOVA >(using ffmanova) and were to find that the interaction term in a 2-way >analysis has borderline significance (say p = 0.045) and a small effect >size, whereas one of the main effects is highly signficant (say p = 6.8e-10) >and has a large effect size. > >Would it in this case be reasonable for me to ignore the interaction term, >and talk only about main effects? And, presuming the main question is fair, >are there general guidlines concerning the relationship between level of >significance and effect size for interaction terms. > >Thank you in advance for your help,===============================================================Robert A. LaBudde, PhD, PAS, Dpl. ACAFS e-mail: ral at lcfltd.com Least Cost Formulations, Ltd. URL: http://lcfltd.com/ 824 Timberlake Drive Tel: 757-467-0954 Virginia Beach, VA 23464-3239 Fax: 757-467-2947 "Vere scire est per causas scire"
Mark Difford
2007-Jul-21 16:07 UTC
[R] Effect size, interactions, and main effects (Stats Question) [ffmanova]
Hi Robert, Thank you for your reply, and for what appears to be some good/sound practical advice on managing this kind of problem. Best Regards, Mark. Robert A LaBudde wrote:> > Statistical significance is "detectability", and depends upon the > size of the sample as well as the effect. A large enough experiment > will result in statistical detectability of almost every interaction > action term allowed. > > This is why estimation, not testing, has become the consensus > recommendation in statistics. > > As a practical matter, evaluate the combined effect of your model > terms with and without the interaction term(s) you are worried about. > Is the reduction in accuracy of physical importance? If so, the > interaction terms are required for scientific reasons. If not, > present both results and indicate the acceptability (for > interpolation) of the simpler model. > > You should also make it your first priority to hypothecate why the > interaction terms are meaningful and expected. If a cause can be > found, it may suggest an alternate model that will eliminate > interactions, or satisfy your anxiety. If not, it may support your > argument to simplify. > > > At 08:58 AM 7/21/2007, Mark wrote: > >>Dear List Members, >> >>I would very much appreciate any pointers you could give me on thefollowing>>matter: >> >>Main Question: >>To what extent does the "rule" that it is unreasonable to talk about main >>effects if there are significant interactions in a model depend uponeffect>>size [of the significant interaction terms]? Or is this not an issue? >> >>More practically: Suppose I were to carry out a so-called Type-II MANOVA >>(using ffmanova) and were to find that the interaction term in a 2-way >>analysis has borderline significance (say p = 0.045) and a small effect >>size, whereas one of the main effects is highly signficant (say p 6.8e-10) >>and has a large effect size. >> >>Would it in this case be reasonable for me to ignore the interaction term, >>and talk only about main effects? And, presuming the main question isfair,>>are there general guidlines concerning the relationship between level of >>significance and effect size for interaction terms. >> >>Thank you in advance for your help, > > ===============================================================> Robert A. LaBudde, PhD, PAS, Dpl. ACAFS e-mail: ral at lcfltd.com > Least Cost Formulations, Ltd. URL: http://lcfltd.com/ > 824 Timberlake Drive Tel: 757-467-0954 > Virginia Beach, VA 23464-3239 Fax: 757-467-2947 > > "Vere scire est per causas scire" > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > >-- View this message in context: http://www.nabble.com/Effect-size%2C-interactions%2C-and-main-effects-%28Stats-Question%29--ffmanova--tf4121771.html#a11723512 Sent from the R help mailing list archive at Nabble.com.