Edison Iglesias de Oliveira
2014-Nov-08 18:34 UTC
[R] Help with sample size calculation for inter-rater reliability study
Dear R masters, I am attempting to calculate the sample size for an inter-rater reliability study using the N2.cohen.kappa function of the irr package. It is a study of 2 raters for a single item with three possible ordinal outcomes. The expected marginal probabilities for those outcomes are 0.2, 0.6 and 0.2. The null hypothesis is that kappa < 0.8 The alternative hypotheis is kappa >=0.8 The following example comes from the irr package manual require(lpSolve) # Testing H0: kappa = 0.4 vs. HA: kappa > 0.4 (=0.6) given that # Marginal Probabilities by two raters are (0.2, 0.25, 0.55). # # one sided test with 80% power: N2.cohen.kappa(c(0.2, 0.25, 0.55), k1=0.6, k0=0.4) # one sided test with 90% power: N2.cohen.kappa(c(0.2, 0.25, 0.55), k1=0.6, k0=0.4, power=0.9) # Marginal Probabilities by two raters are (0.2, 0.05, 0.2, 0.05, 0.2, 0.3) # Testing H0: kappa = 0.1 vs. HA: kappa > 0.1 (=0.5) given that # # one sided test with 80% power: N2.cohen.kappa(c(0.2, 0.05, 0.2, 0.05, 0.2, 0.3), k1=0.5, k0=0.1) In my case I would be testing H0: kappa < 0.8 vs HA: kappa >= 08 given Marginal probabilities by two raters as (0.2, 0.6, 0.2) However the following argument will not work N2.cohen.kappa (mrg=c(0.2,0.6,0.2), k1>=0.8, k0<0.8, alpha=0.05, power= 0.8, twosided=FALSE) I have also tried the kappaSize package without success Power3Cats(kappa0<0.8, kappa1>=0.8, props=c(0.2,0.6,0.2), raters=2, alpha=0.05, power=0.80) Can anyone offer me some guidance? Best regards, Edison [[alternative HTML version deleted]]