Dear R-help Team, I am from India and have a query on 'N.cohen.kappa' Sample size calculations for Cohen's Kappa Statistic. I have calculated manually the sample size using the formula mentioned in "Cantor, A. B. (1996) Sample-size calculation for Cohen?s kappa. Psychological Methods, 1, 150- 153". Later came to know that it can be done using the R 'irr' package. I got a different number. Let us consider the following two situations: For situation 1, the sample size is 1370 using R Testing H0: kappa = 0.81 vs. HA: kappa> 0.95 given that kappa = 0.95 and both raters classify 1.5% of subjects as positive. R command used is: N.cohen.kappa(0.015, 0.015, 0.95, 0.81, alpha=0.05, power=0.8, twosided=FALSE ). But for the same situation, the sample size is much higher by manual calculation, which is 8580. For situation 2, the sample size is 74 by using R and is matching with the manual calculation too. Testing H0: kappa = 0.81 vs. HA: kappa> 0.95 given that kappa = 0.95 and rater1 classify 40% of subjects and rate2 classify 50% of subjects as positive. R command used is: N.cohen.kappa(0.40, 0.50, 0.95, 0.81, alpha=0.05, power=0.8, twosided=FALSE ). I am attaching both the 'Excel sheet formula-kappa sample size situation1 & 2'). Why is this so? Please help me to sort this out. Looking forward to hearing from you. Best, Kalaivani -- *Dr. Kalaivani Mani, * *M.Sc., Biostatistics (CMC, Vellore), Ph.D. (AIIMS, Delhi)* *Scientist-IV* *Dept. of Biostatistics* *All India Institute of Medical Sciences* *New Delhi-110029, India.* *Mobile:91-9717319082*
R is a fully capable computation environment... you do not have to use any contributed package to implement an algorithm. It may be convenient to rely on such resources, but you must always be aware that such code may not be correct. If you think it is not correct, then you can certainly express your validation calculations in R, and if you share such information with the maintainer of that package then they may either correct their package or suggest what considerations you may have neglected. However, we participants on this list who happen to use R may or may not know anything about your algorithm... chances are pretty high that we don't. If you express your concerns using a reproducible R code example [1] we might take an interest... but debugging your Excel file is pretty unlikely to be of interest here (in addition to being hard to share... the mailing list strips most attachments). As the Posting Guide says, you really should be corresponding with the package maintainer. [2] [1] https://cran.r-project.org/web/packages/reprex/vignettes/reprex-dos-and-donts.html [2] https://cran.r-project.org/web/packages/irr/index.html On June 16, 2022 2:42:31 AM PDT, Kalaivani Mani <manikalaivani at gmail.com> wrote:>Dear R-help Team, > >I am from India and have a query on 'N.cohen.kappa' Sample size >calculations for Cohen's Kappa Statistic. I have calculated manually the >sample size using the formula mentioned in "Cantor, A. B. (1996) >Sample-size calculation for Cohen?s kappa. Psychological Methods, 1, 150- >153". Later came to know that it can be done using the R 'irr' package. I >got a different number. > >Let us consider the following two situations: > >For situation 1, the sample size is 1370 using R >Testing H0: kappa = 0.81 vs. HA: kappa> 0.95 given that kappa = 0.95 and >both raters classify 1.5% of subjects as positive. >R command used is: N.cohen.kappa(0.015, 0.015, 0.95, 0.81, alpha=0.05, >power=0.8, twosided=FALSE ). > >But for the same situation, the sample size is much higher by manual >calculation, which is 8580. > >For situation 2, the sample size is 74 by using R and is matching with the >manual calculation too. >Testing H0: kappa = 0.81 vs. HA: kappa> 0.95 given that kappa = 0.95 and >rater1 classify 40% of subjects and rate2 classify 50% of subjects as >positive. >R command used is: N.cohen.kappa(0.40, 0.50, 0.95, 0.81, alpha=0.05, >power=0.8, twosided=FALSE ). > >I am attaching both the 'Excel sheet formula-kappa sample size situation1 & >2'). > >Why is this so? Please help me to sort this out. > >Looking forward to hearing from you. > >Best, >Kalaivani >-- Sent from my phone. Please excuse my brevity.
Hi Kalaivani, The N.cohen.kappa function was written by Matthais Gamer, the maintainer of the irr package. Both that function and N2.cohen.kappa (written by Puspendra Singh) involve corrections that are described in the references on the respective help pages. It is likely that there will be small differences in the estimates for large N in the different methods of calculation. I cannot advise which would best suit your purpose as I only did testing and refining code in the N2.cohen.kappa function. Perhaps corresponding directly with Matthais Gamer would be your best option. Jim On Fri, Jun 17, 2022 at 2:20 AM Kalaivani Mani <manikalaivani at gmail.com> wrote:> > Dear R-help Team, > > I am from India and have a query on 'N.cohen.kappa' Sample size > calculations for Cohen's Kappa Statistic. I have calculated manually the > sample size using the formula mentioned in "Cantor, A. B. (1996) > Sample-size calculation for Cohen?s kappa. Psychological Methods, 1, 150- > 153". Later came to know that it can be done using the R 'irr' package. I > got a different number. > > Let us consider the following two situations: > > For situation 1, the sample size is 1370 using R > Testing H0: kappa = 0.81 vs. HA: kappa> 0.95 given that kappa = 0.95 and > both raters classify 1.5% of subjects as positive. > R command used is: N.cohen.kappa(0.015, 0.015, 0.95, 0.81, alpha=0.05, > power=0.8, twosided=FALSE ). > > But for the same situation, the sample size is much higher by manual > calculation, which is 8580. > > For situation 2, the sample size is 74 by using R and is matching with the > manual calculation too. > Testing H0: kappa = 0.81 vs. HA: kappa> 0.95 given that kappa = 0.95 and > rater1 classify 40% of subjects and rate2 classify 50% of subjects as > positive. > R command used is: N.cohen.kappa(0.40, 0.50, 0.95, 0.81, alpha=0.05, > power=0.8, twosided=FALSE ). > > I am attaching both the 'Excel sheet formula-kappa sample size situation1 & > 2'). > > Why is this so? Please help me to sort this out. > > Looking forward to hearing from you. > > Best, > Kalaivani > > -- > *Dr. Kalaivani Mani, * > > *M.Sc., Biostatistics (CMC, Vellore), Ph.D. (AIIMS, Delhi)* > > *Scientist-IV* > > *Dept. of Biostatistics* > > *All India Institute of Medical Sciences* > *New Delhi-110029, India.* > *Mobile:91-9717319082* > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.