Edison Iglesias de Oliveira
2014-Nov-08  18:34 UTC
[R] Help with sample size calculation for inter-rater reliability study
Dear R masters,
I am attempting to calculate the sample size for an inter-rater reliability
study using the N2.cohen.kappa function of the irr package. 
It is a study of 2 raters for a single item with three possible ordinal
outcomes. The expected marginal probabilities for those outcomes are 0.2,
0.6 and 0.2. 
The null hypothesis is that kappa < 0.8
The alternative hypotheis is kappa >=0.8
The following example comes from the irr package manual
require(lpSolve)
     # Testing H0: kappa = 0.4 vs. HA: kappa > 0.4 (=0.6) given that
     # Marginal Probabilities by two raters are (0.2, 0.25, 0.55).
     #
     # one sided test with 80% power:
     N2.cohen.kappa(c(0.2, 0.25, 0.55), k1=0.6, k0=0.4)
     # one sided test with 90% power:
     N2.cohen.kappa(c(0.2, 0.25, 0.55), k1=0.6, k0=0.4, power=0.9)
     # Marginal Probabilities by two raters are (0.2, 0.05, 0.2, 0.05, 0.2,
0.3)
     # Testing H0: kappa = 0.1 vs. HA: kappa > 0.1 (=0.5) given that
     #
     # one sided test with 80% power:
     N2.cohen.kappa(c(0.2, 0.05, 0.2, 0.05, 0.2, 0.3), k1=0.5, k0=0.1)
In my case I would be testing H0: kappa < 0.8 vs HA: kappa >= 08 given
Marginal probabilities by two raters as (0.2, 0.6, 0.2)
However the following argument will not work
N2.cohen.kappa (mrg=c(0.2,0.6,0.2), k1>=0.8, k0<0.8, alpha=0.05, power=
0.8,
twosided=FALSE)
I have also tried the kappaSize package without success
Power3Cats(kappa0<0.8, kappa1>=0.8, props=c(0.2,0.6,0.2), raters=2,
alpha=0.05, power=0.80)
Can anyone offer me some guidance?
Best regards,
Edison 		 	   		   		 	   		  
	[[alternative HTML version deleted]]