R help - Jun 2013 - Post hoc power analysis for mixed-effects models

Dear all,

I have been searching ways to run power analysis for mixed-effects models.
However, I have not been successful
in the research. Today I would like to ask your help. As long as I see from my
search, Martin Julien wrote a package
called pamm for the power analysis. One of the limitations in the current
version is that pamm cannot handle
categorical fixed variables. Todd Jobes introduced his script to run power
analysis for mixed-effects models
(http://toddjobe.blogspot.co.nz/2009/09/power-analysis-for-mixed-effect-models.html).
However, some parts of
the script is beyond my knowledge. I am not sure if I can run power analysis
with categorical variables either. Is there
anybody who has run post hoc power analysis for mixed-effects models? If you
have experiences, I would like to
ask your help. Thank you very much for taking your time.

Yours,
Kota

This email may be confidential and subject to legal privilege, it may
not reflect the views of the University of Canterbury, and it is not
guaranteed to be virus free. If you are not an intended recipient,
please notify the sender immediately and erase all copies of the message
and any attachments.

Please refer to http://www.canterbury.ac.nz/emaildisclaimer for more
information.

	[[alternative HTML version deleted]]

On Jun 4, 2013, at 4:46 PM, Kota Hattori wrote:
> Dear all,
>
> I have been searching ways to run power analysis for mixed-effects  
> models. However, I have not been successful
> in the research. Today I would like to ask your help. As long as I  
> see from my search, Martin Julien wrote a package
> called pamm for the power analysis. One of the limitations in the  
> current version is that pamm cannot handle
> categorical fixed variables. Todd Jobes introduced his script to run  
> power analysis for mixed-effects models
>
(http://toddjobe.blogspot.co.nz/2009/09/power-analysis-for-mixed-effect-models.html
> ). However, some parts of
> the script is beyond my knowledge. I am not sure if I can run power  
> analysis with categorical variables either. Is there
> anybody who has run post hoc power analysis for mixed-effects  
> models? If you have experiences, I would like to
> ask your help. Thank you very much for taking your time.
Can you provide a sensible justification for "post hoc power analysis?  
I know the terminology has crept into widespread use due to its  
existence in either SAS or SPSS (I forget which), but I have doubts  
about its validity. It mixes up the order of statistical testing  
logic. Power analysis is something done _before_ the study. If a  
statistical procedure is done after a study's data is collected with  
the very dubious assumption that the sample statistics are the  
population statistics, it's not a power analysis.

-- 
David Winsemius, MD
Alameda, CA, USA

Kota Hattori <kota.hattori <at> canterbury.ac.nz> writes:
>  Dear all, I have been searching ways to run power analysis for
> mixed-effects models. However, I have not been successful in the
> research. Today I would like to ask your help. As long as I see from
> my search, Martin Julien wrote a package called pamm for the power
> analysis. One of the limitations in the current version is that pamm
> cannot handle categorical fixed variables. Todd Jobes introduced his
> script to run power analysis for mixed-effects models
> (http://toddjobe.blogspot.co.nz/2009/09/
    power-analysis-for-mixed-effect-models.html).
> However, some parts of the script is beyond my knowledge. I am not
> sure if I can run power analysis with categorical variables
> either. Is there anybody who has run post hoc power analysis for
> mixed-effects models? If you have experiences, I would like to ask
> your help. Thank you very much for taking your time.
  I share David Winsemius's concerns about post hoc power analysis:
this thread from 2008 gives some important reading.
<http://comments.gmane.org/gmane.comp.lang.r.ecology/472>

  The script you reference is a pretty generic introduction
to simulating data corresponding to a fixed + random effects
structure, so it should certainly be adaptable to categorical
variables.  

  This doesn't immediately solve your problem, but you might
work through chapter 5 of
http://www.math.mcmaster.ca/~bolker/emdbook/book.pdf
to strengthen your background knowledge ...

  Further mixed-model-relevant questions should probably
go to r-sig-mixed-models at r-project.org.

  Ben Bolker