Hi All, I am using fleiss kappa for inter rater agreement. Are there any know issues with Fleiss kappa calculation in R? Even when I supply mock data with total agreement among the raters I do not get a kappa value of 1. instead I am getting negative values. I am using the irr package version 0.70 Any help is much appreciated. Thanks and Regards M [[alternative HTML version deleted]]
If you apply the function to a simple dataframe or show your code, you might be able to get more accurate help. I've used the IRR package in the past and haven't noticed any problems (maybe I overlooked them ?) david freedman mehdi ebrahim wrote:> > Hi All, > > I am using fleiss kappa for inter rater agreement. Are there any know > issues with Fleiss kappa calculation in R? Even when I supply mock data > with total agreement among the raters I do not get a kappa value of 1. > instead I am getting negative values. > > I am using the irr package version 0.70 > > Any help is much appreciated. > > Thanks and Regards > > M > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > >-- View this message in context: http://www.nabble.com/Help-With-Fleiss-Kappa-tp24456963p24461821.html Sent from the R help mailing list archive at Nabble.com.
At 6:48 PM +1200 7/13/09, mehdi ebrahim wrote:>Hi All, > >I am using fleiss kappa for inter rater agreement. Are there any know >issues with Fleiss kappa calculation in R? Even when I supply mock data >with total agreement among the raters I do not get a kappa value of 1. >instead I am getting negative values. > >I am using the irr package version 0.70 > >Any help is much appreciated.Mehdi, Are you by any chance giving the function the agreement matrix rather than the raw data? The kappam.fliess function seems to want the ratings rather than the agreement matrix. Bill -- William Revelle http://personality-project.org/revelle.html Professor http://personality-project.org/personality.html Department of Psychology http://www.wcas.northwestern.edu/psych/ Northwestern University http://www.northwestern.edu/ Attend ISSID/ARP:2009 http://issid.org/issid.2009/
Hi All,
I am using raw data.
Following is an example of my data frame
P1 P2 P3
H 1 1 1
M 2 4 2
L 2 2 2
H,M,L relate to the three cases, and P1 to P3 relate to the
raters/participants.
The problem I am having is if all the values are 1 in the above data frame
(say I have named it r1.df) and I use the following command
kappam.fleiss(r1.df) I get a negative kappa value.
Thanks and Regards
Mehdi E
On Tue, Jul 14, 2009 at 2:09 AM, William Revelle <lists@revelle.net>
wrote:
> At 6:48 PM +1200 7/13/09, mehdi ebrahim wrote:
>
>> Hi All,
>>
>> I am using fleiss kappa for inter rater agreement. Are there any know
>> issues with Fleiss kappa calculation in R? Even when I supply mock
data
>> with total agreement among the raters I do not get a kappa value of 1.
>> instead I am getting negative values.
>>
>> I am using the irr package version 0.70
>>
>> Any help is much appreciated.
>>
>
> Mehdi,
> Are you by any chance giving the function the agreement matrix rather than
> the raw data? The kappam.fliess function seems to want the ratings rather
> than the agreement matrix.
>
> Bill
>
>
> --
> William Revelle http://personality-project.org/revelle.html
> Professor
> http://personality-project.org/personality.html
> Department of Psychology
> http://www.wcas.northwestern.edu/psych/
> Northwestern University http://www.northwestern.edu/
> Attend ISSID/ARP:2009 http://issid.org/issid.2009/
>
--
Mehdi
(You want something? Go Get It! . - A pursuit for happiness )
[[alternative HTML version deleted]]