R help - Jan 2022 - Repeatability Analysis of Ordinal Data

Greetings, Colleagues:

I have several Likert-type ordinal data sets consisting of animal responses with
repeated measures. I was able to implement a CLMM model easily enough with the
package `ordinal`. However, the package does not support repeatability analyses.
 
Assuming that I subset my data according to treatment and/or sex, I am keen to
try the `ordinalRR` package. According to the package documentation
(https://cran.r-project.org/web/packages/ordinalRR/ordinalRR.pdf), performing
`summary()` on the output from the function `ordinalRR()` returns the point
estimates for each rater and for each pairwise combination of raters. However,
is it possible to return an overall repeatability value and a 95% credible
interval across all raters?

What follows is a stock procedure from the package reference document:

#-------------------------------------------------------------------------------
library(ordinalRR)

# load the dataset that comes with the package
data(followup)

# preprocess data to accommodate the package functions
followup.pre <- preprocess(followup)

# perform the analysis
followup.random <- ordinalRR(followup.pre)

summary(followup.random)

Call:
  ordinalRR(followup.pre)

Data: 30 parts, 3 operators, 2 repetitions with 4 ordinal categories.
Random-effects model MCMC chain: 1000 burn-in and 10000 retained.

Simple repeatability and model parameter estimates by rater:
  Rater j Repeatability  a_j d_{j,1} d_{j,2} d_{j,3}
1         0.900 12.0    -1.5    -0.1     0.6
2         0.900 10.9    -1.6    -0.3     0.5
3         0.933 12.7    -1.5    -0.2     0.5

Simple repeatability and reproducibility (R&R) point estimates for pairs of
raters:
  Rater j Rater j' (R&R)_{j,j'}
1        2        0.808
1        3        0.900
2        3        0.850
#-------------------------------------------------------------------------------

Kind Regards,
Salvatore Sidoti
PhD Candidate
The Ohio State University
Columbus, Ohio USA

Package-specific questions for non-standard packages (see
https://cran.r-project.org/doc/FAQ/R-FAQ.html#Add-on-packages-in-R for
the list of current standard packages) are generally off topic here
per the posting guide (linked below). It is suggested there that you
contact the package maintainer (?maintainer) for such questions.

Bert Gunter

"The trouble with having an open mind is that people keep coming along
and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Tue, Feb 1, 2022 at 2:02 AM Sidoti, Salvatore <sidoti.23 at osu.edu>
wrote:
>
> Greetings, Colleagues:
>
> I have several Likert-type ordinal data sets consisting of animal responses
with repeated measures. I was able to implement a CLMM model easily enough with
the package `ordinal`. However, the package does not support repeatability
analyses.
>
> Assuming that I subset my data according to treatment and/or sex, I am keen
to try the `ordinalRR` package. According to the package documentation
(https://cran.r-project.org/web/packages/ordinalRR/ordinalRR.pdf), performing
`summary()` on the output from the function `ordinalRR()` returns the point
estimates for each rater and for each pairwise combination of raters. However,
is it possible to return an overall repeatability value and a 95% credible
interval across all raters?
>
> What follows is a stock procedure from the package reference document:
>
>
#-------------------------------------------------------------------------------
> library(ordinalRR)
>
> # load the dataset that comes with the package
> data(followup)
>
> # preprocess data to accommodate the package functions
> followup.pre <- preprocess(followup)
>
> # perform the analysis
> followup.random <- ordinalRR(followup.pre)
>
> summary(followup.random)
>
> Call:
>   ordinalRR(followup.pre)
>
> Data: 30 parts, 3 operators, 2 repetitions with 4 ordinal categories.
> Random-effects model MCMC chain: 1000 burn-in and 10000 retained.
>
> Simple repeatability and model parameter estimates by rater:
>   Rater j Repeatability  a_j d_{j,1} d_{j,2} d_{j,3}
> 1         0.900 12.0    -1.5    -0.1     0.6
> 2         0.900 10.9    -1.6    -0.3     0.5
> 3         0.933 12.7    -1.5    -0.2     0.5
>
> Simple repeatability and reproducibility (R&R) point estimates for
pairs of raters:
>   Rater j Rater j' (R&R)_{j,j'}
> 1        2        0.808
> 1        3        0.900
> 2        3        0.850
>
#-------------------------------------------------------------------------------
>
> Kind Regards,
> Salvatore Sidoti
> PhD Candidate
> The Ohio State University
> Columbus, Ohio USA
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.