R help - Jun 2010 - shrout & fleiss ICC´s with varying numbers of judges

Win7, R2.11.0

I am working on a report together with several co-authors. The data
concern several performance measures on a set of groups. These measures
are scored by external judges. We report findings on several datasets,
including several of the 6 ICC's discussed by Shrout & Fleiss (1979). We
determine these using the icc function from the irr package. We have
also used the ICC function from the psych package.

However, one of the datasets has varying numbers of judges for each
group. One group may be rated by 2 judges, whereas others may even have
ratings by up to 5 judges. My problem is that I don't see how I can use
the icc or ICC functions for determining the icc's these groups, since
icc/ICC require that all groups have the same number of judges (as far
as I understand). Alternatively, I could report the ICC1 and ICC2 (in
the multilevel package), but these are different from the Shrout &
Fleiss ones, and we would like to be consistent. (of course, I know that
not all 6 icc's apply in every case, but we would like to stay within
the Shrout & Fleiss framework and then report the relevant icc). I have
searched extensively for a function that would allow to calculate S&F
icc's for a varying number of judges, but I can't find one. Are there
any there functions that allow me to do this? Or is there a way to still
use icc or ICC?

thanks,
Peter Verbeet



<P><p><font face="Arial, Helvetica, sans-serif"
size="2"
style="font-size:13.5px">_______________________________________________________________<BR>Care2
makes it easy for everyone to live a healthy, green lifestyle
and impact the causes you care about most. Over 12 Million members!
http://www.care2.com

Feed a child by searching the web! Learn how
http://www.care2.com/toolbar</font>
	[[alternative HTML version deleted]]

hi Peter,

There's no single function for ICC with variable number of judges. To
estimate the variances in that case you need hierarchical linear modeling. I
posted code for this at Stackoverflow in answer to your question there:

  http://stackoverflow.com/questions/3205176/

Stats questions get answered rarely on Stackoverflow but you may want to try
the new site focused on statistics:
  http://area51.stackexchange.com/proposals/33/statistical-analysis


hope that helps,
Turadg Aleahmad

-- 
View this message in context:
http://r.789695.n4.nabble.com/shrout-fleiss-ICC-s-with-varying-numbers-of-judges-tp2257858p2323230.html
Sent from the R help mailing list archive at Nabble.com.