R help - Sep 2008 - Proper power computation for one-sided binomial tests.

Hi, I trying to determine the best way to compute the power for a
one-sample one-sided binomial test.  Specifically I need to sample a
population of individuals and ask whether a sample rate of 0% is
compatable with a minimum threshold of 3% and how many samples are needed.

I have made use of power.prop.test but I am not sure if a) that is the
correct (or best) function to use and b) if the output is quite right.

Here is a sample run:
> power.prop.test(p1=0, p2=0.03, sig.level=0.05, power=0.90,
alt="one.sided")

     Two-sample comparison of proportions power calculation

              n = 279.3004
             p1 = 0
             p2 = 0.03
      sig.level = 0.05
          power = 0.9
    alternative = one.sided

 NOTE: n is number in *each* group

This is an attempt to test whether a sample of 0% occurrance is compatable
with an a-priori probability of 3% at the specified significance levels.

My questions are those above, and, as a followup whether the caveat about
n being the number in each group means that I need to sample twice that
number in a single group.  I don't believe so but I want to be sure.

	Thanks in advance,
	Collin Lynch.

Collin Lynch wrote:
> Hi, I trying to determine the best way to compute the power for a
> one-sample one-sided binomial test.  Specifically I need to sample a
> population of individuals and ask whether a sample rate of 0% is
> compatable with a minimum threshold of 3% and how many samples are needed.
>
> I have made use of power.prop.test but I am not sure if a) that is the
> correct (or best) function to use and b) if the output is quite right.
>
> Here is a sample run:
>   
>> power.prop.test(p1=0, p2=0.03, sig.level=0.05, power=0.90,
>>     
> alt="one.sided")
>
>      Two-sample comparison of proportions power calculation
>
>               n = 279.3004
>              p1 = 0
>              p2 = 0.03
>       sig.level = 0.05
>           power = 0.9
>     alternative = one.sided
>
>  NOTE: n is number in *each* group
>
> This is an attempt to test whether a sample of 0% occurrance is compatable
> with an a-priori probability of 3% at the specified significance levels.
>
> My questions are those above, and, as a followup whether the caveat about
> n being the number in each group means that I need to sample twice that
> number in a single group.  I don't believe so but I want to be sure.
>
>   
Yes, that's wrong. Now you can be sure ;-)

For this kind of problem I'd go directly for the binomial distribution. 
If the actual probability is 0, this is essentially deterministic and 
you can look at

 > binom.test(0,99,p=.03, alt="less")

    Exact binomial test

data:  0 and 99
number of successes = 0, number of trials = 99, p-value = 0.04902
alternative hypothesis: true probability of success is less than 0.03
95 percent confidence interval:
 0.00000000 0.02980667
sample estimates:
probability of success
                     0

So you have significance at n=99, which I think we can easily agree is 
less than two times 249....


-- 
   O__  ---- Peter Dalgaard             ?ster Farimagsgade 5, Entr.B
  c/ /'_ --- Dept. of Biostatistics     PO Box 2099, 1014 Cph. K
 (*) \(*) -- University of Copenhagen   Denmark      Ph:  (+45) 35327918
~~~~~~~~~~ - (p.dalgaard at biostat.ku.dk)              FAX: (+45) 35327907

Thank you Peter.  That is incredibly helpfyul, and much much smaller!

	Best,
	Collin.

Am 23.09.2008 um 23:57 schrieb Peter Dalgaard:
> For this kind of problem I'd go directly for the binomial
> distribution. If the actual probability is 0, this is essentially
> deterministic and you can look at
>
> > binom.test(0,99,p=.03, alt="less")
>
> This means that you don't sample from the p=.03 population?
> Note that there is a 5 per cent chance to have 0 failures in 99
> trials with p=.03.
In this case I am given the p=.03 as static value.  So my goal is to
compare the sample statistic against that.  I am essentially checking the
real population against an a-priori hypothesis of p=.03 or rather whatever
we set it at (long story).

	Collin.

>
> Am 23.09.2008 um 23:57 schrieb Peter Dalgaard:
>
>> For this kind of problem I'd go directly for the binomial
>> distribution. If the actual probability is 0, this is essentially
>> deterministic and you can look at
>>
>> > binom.test(0,99,p=.03, alt="less")
>>
>
> > This means that you don't sample from the p=.03 population?
>>  Note that there is a 5 per cent chance to have 0 failures in 99
> > trials with p=.03.
> Yes, that's what I read the task as saying: Sample from p=0.00 when the
> hypothesis is p=0.03. Then rejection happens with probability 1 when n
>  >= 99. Actually, he said that we could assume the _sample_ rate to be
> 0%, but that is only assured when p=0.0.
>
> (You can continue the game by looking at the probability of getting 0
> failures, depending on the true p. E.g., if p=0.001, we have
>
>  > dbinom(0, 99, 0.001)
> [1] 0.9056978
>
> i.e. 90% power to detect at 5% level. And further continue into a full
> power analysis where you calculate the probability of a failure rate
> that is significantly different from 0.03 depending on p and n.)

So my task is to compare a single sample population (which should exhibit
0 successes, against an a-priori p value with the goal of asking if the
sample is consistent with a p-value of that or less.

My goal is to determine the number of sample points needed to state with
95% confidence that the sample is predictive at alpha=.05.  The process in
question should always return 0 successes but I need to know how many to
test so as to make those 0 successes meaningful.

If I understand you correctly Peter then the binom.test procedure with a
specified p value will do.  Or rather that:
> binom.test(0, 99, p=0.01, alt="less", conf.level=0.99)
        Exact binomial test

data:  0 and 99
number of successes = 0, number of trials = 99, p-value = 0.3697
alternative hypothesis: true probability of success is less than 0.01
99 percent confidence interval:
 0.00000000 0.04545154
sample estimates:
probability of success
                     0

correctly informs me that at a 99% confidence interval 99 systems is
woefully inadequate.

      Thanks,
      Collin Lynch.




       Thanks,
       Collin Lynch.