R help - Jun 2007 - Power calculation with measurement error

Hi all,

Hopefully this will be quick, I'm looking for pointers to packages/ 
functions that would allow me to calculate the power of a t.test when  
the DV has measurement error. That is, I understand that, ceteris  
paribus, experiments using measure with more error (lower  
reliability) will have lower power.

Mike

--
Mike Lawrence
Graduate Student, Department of Psychology, Dalhousie University

Website: http://memetic.ca

Public calendar: http://icalx.com/public/informavore/Public

"The road to wisdom? Well, it's plain and simple to express:
Err and err and err again, but less and less and less."
	- Piet Hein

I don't know of a current package that does this (others may), but if
you know what you expect your data to look like you can simulate it and
calculate power that way.

Basically, write a function that will simulate data with the level of
measurement error that you expect in the real data (or have the amount
of measurement error passed in as a parameter so you can examine the
effect of diffenent values).  Then have the function compute the t test
(or other test that you plan to do) and return the p-value from the
test.

Then you can simulate the process with a command like:
> out1 <- replicate( 1000, myfunction(n=25, err=.1, diff=.5) )
And compute the power with:
> mean( out1 < 0.05 ) # or whatever alpha value you want.
Hope this helps,

-- 
Gregory (Greg) L. Snow Ph.D.
Statistical Data Center
Intermountain Healthcare
greg.snow at intermountainmail.org
(801) 408-8111
 
 
> -----Original Message-----
> From: r-help-bounces at stat.math.ethz.ch 
> [mailto:r-help-bounces at stat.math.ethz.ch] On Behalf Of Mike Lawrence
> Sent: Tuesday, June 26, 2007 5:13 AM
> To: Rhelp
> Subject: [R] Power calculation with measurement error
> 
> Hi all,
> 
> Hopefully this will be quick, I'm looking for pointers to 
> packages/ functions that would allow me to calculate the 
> power of a t.test when the DV has measurement error. That is, 
> I understand that, ceteris paribus, experiments using measure 
> with more error (lower
> reliability) will have lower power.
> 
> Mike
> 
> --
> Mike Lawrence
> Graduate Student, Department of Psychology, Dalhousie University
> 
> Website: http://memetic.ca
> 
> Public calendar: http://icalx.com/public/informavore/Public
> 
> "The road to wisdom? Well, it's plain and simple to express:
> Err and err and err again, but less and less and less."
> 	- Piet Hein
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide 
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

On 26-Jun-07, at 8:12 AM, Mike Lawrence wrote:
> Hi all,
> Hopefully this will be quick, I'm looking for pointers to packages/
> functions that would allow me to calculate the power of a t.test when
> the DV has measurement error. That is, I understand that, ceteris
> paribus, experiments using measure with more error (lower
> reliability) will have lower power.
I came across a reference (http://memetic.ca/reliability.pdf) that  
provides a formula for calculating the noncentrality parameter for  
tests using imperfect measures (see Eq. 4), as well as a table of  
some resulting power estimates. However, while I have created a (very  
slow) monte carlo function that so far as I can tell matches their  
results, when I attempt to implement their analytic solution it's way  
off. Can anyone see what I'm doing incorrectly?

n=100
r=.5 #reliability
e=.5 #effect size
delta=(sqrt(r*n)/2)*e
power.t.test(n,delta,sig.level=.05,alternative='one.sided')

      Two-sample t test power calculation

               n = 100
           delta = 1.767767
              sd = 1
       sig.level = 0.05
           power = 1
     alternative = one.sided

NOTE: n is number in *each* group


Meanwhile, their tables and my monte carlo method say that the power  
in that circumstance should be .7

Help?

Mike