R help - Jun 2006 - individual scales in random subset of pairwise distance survey

Hello,

I'm curious if anyone has encounted a version of this
problem
(and it's solution) involving finding a consistent set
of scales
for subsets of survey data.

The goal is to obtain peoples' rankings of pairwise
similarity of a large
number of items, on a 1..5 scale for example, and
average these
across people to use as input to MDS:
  How similar is object A to B    on a 1..5 scale ___
  How similar is object A to C    on a 1..5 scale ___
etc.
  
Because there are many items, there are N(N-1)/2
pairs, so it is not
practical to show every pair to everyone.   Showing
people the
pairs corresponding to random subsets of the objects
seems desirable.

THe problem is that, a particular random subset might
by chance
contain objects that would all be rated "5" if one
were to see
the entire dataset.  When ranking pairs from this
subset, the scale
of 1..5 is different.  

If we ensure that each pair of people must see some
data in common,
then one can think about obtaining a set of scales,
one for each
person, that causes the data that is commonly ranked
to have
as similar scores as possible, summed across all pairs
of people.


Please let me know if you know of a standard procedure
for
this or any similar problems.    

Thank you.

Perhaps you could try clustering the objects.

# generate test data
set.seed(1)
n <- 25 # number of items
mat <- matrix(0, n, n)
# use different labelling scheme if > 26 items
rownames(mat) <- colnames(mat) <- letters[1:n]
mat[lower.tri(mat)] <- sample(5, n * (n-1)/2, TRUE)
mat <- mat + t(mat) + diag(1, n)

# cluster and plot
plot(hclust(as.dist(mat)))


On 6/15/06, context grey <mobygeek at yahoo.com>
wrote:
>
> Hello,
>
> I'm curious if anyone has encounted a version of this
> problem
> (and it's solution) involving finding a consistent set
> of scales
> for subsets of survey data.
>
> The goal is to obtain peoples' rankings of pairwise
> similarity of a large
> number of items, on a 1..5 scale for example, and
> average these
> across people to use as input to MDS:
>  How similar is object A to B    on a 1..5 scale ___
>  How similar is object A to C    on a 1..5 scale ___
> etc.
>
> Because there are many items, there are N(N-1)/2
> pairs, so it is not
> practical to show every pair to everyone.   Showing
> people the
> pairs corresponding to random subsets of the objects
> seems desirable.
>
> THe problem is that, a particular random subset might
> by chance
> contain objects that would all be rated "5" if one
> were to see
> the entire dataset.  When ranking pairs from this
> subset, the scale
> of 1..5 is different.
>
> If we ensure that each pair of people must see some
> data in common,
> then one can think about obtaining a set of scales,
> one for each
> person, that causes the data that is commonly ranked
> to have
> as similar scores as possible, summed across all pairs
> of people.
>
>
> Please let me know if you know of a standard procedure
> for
> this or any similar problems.
>
> Thank you.
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide!
http://www.R-project.org/posting-guide.html
>