R help - Jul 2005 - Proportion test in three-chices experiment

Hi,

I wish to analyze with R the results of a perception experiment in which
subjects had to recognize each stimulus among three choices (this was a
forced-choice design).  The experiment runs under two different
conditions and the data is like the following:

   N1 : count of trials in condition 1
   p11, p12, p13: proportions of choices 1, 2, and 3 in condition 1
   
   N2 : count of trials in condition 2
   p21, p22, p23: proportions of choices 1, 2, and 3 in condition 2
   
How can I test whether the triple (p11,p12,p13) is different from the
triple (p21,p22,p23)?  Clearly, prop.test does not help me here, because
it relates to two-choices tests.

I apologize if the answer is trivial, but I am relatively new to R and
could not find any pointers in the FAQ or in the mailing list archives.

Thanks in advance for any help,

-- 
Rafael Laboissiere

Have you considered "BTm" in library(BradleyTerry)?  Consider the 
following example:

 > cond1 <- data.frame(winner=rep(LETTERS[1:3], e=2),
+          
loser=c("B","C","A","C","A","B"),
+           Freq=1:6)
 > cond2 <- data.frame(winner=rep(LETTERS[1:3], e=2),
+          
loser=c("B","C","A","C","A","B"),
+           Freq=6:1)
 > fit1 <- BTm(cond1~..)
 > fit2 <- BTm(cond2~..)
 > fit12 <- BTm(rbind(cond1, cond2)~..)
 > Dev12 <- (fit1$deviance+fit2$deviance
+           -fit12$deviance)
 > pchisq(Dev12, 2, lower=FALSE)
[1] 0.8660497

	  This says the difference between the two data sets, cond1 and cond2, 
are not statistically significant.

	  Do you present each subject with onely one pair?  If yes, then this 
model is appropriate.  If no, then the multiple judgments by the same 
subject are not statistically independent, as assumed by this model. 
However, if you don't get statistical significance via this kind of 
computation, it's unlikely that a better model would give you 
statistical significance.  If you get a p value of, say, 0.04, then the 
difference is probably NOT statistically significant.

	  The p value you get here would be an upper bound.  You could get a 
lower bound by using only one of the three pairs presented to each 
subject selected at random.  If that p value were statistically 
significant, then I think it is safe to say that your two sets of 
conditions are significantly different.  For any value in between, it 
would depend on how independent the three choices by the same subject. 
You might, for example, delete one of the three pairs at random and use 
the result of that comparison.

	  There are doubtless better techniques, but I'm not familiar with 
them.  Perhaps someone else will reply to my reply.

	  spencer graves

Rafael Laboissiere wrote:
> Hi,
> 
> I wish to analyze with R the results of a perception experiment in which
> subjects had to recognize each stimulus among three choices (this was a
> forced-choice design).  The experiment runs under two different
> conditions and the data is like the following:
> 
>    N1 : count of trials in condition 1
>    p11, p12, p13: proportions of choices 1, 2, and 3 in condition 1
>    
>    N2 : count of trials in condition 2
>    p21, p22, p23: proportions of choices 1, 2, and 3 in condition 2
>    
> How can I test whether the triple (p11,p12,p13) is different from the
> triple (p21,p22,p23)?  Clearly, prop.test does not help me here, because
> it relates to two-choices tests.
> 
> I apologize if the answer is trivial, but I am relatively new to R and
> could not find any pointers in the FAQ or in the mailing list archives.
> 
> Thanks in advance for any help,
> 
-- 
Spencer Graves, PhD
Senior Development Engineer
PDF Solutions, Inc.
333 West San Carlos Street Suite 700
San Jose, CA 95110, USA

spencer.graves at pdf.com
www.pdf.com <http://www.pdf.com>
Tel:  408-938-4420
Fax: 408-280-7915

Rafael,

when testing binomial hypotheses with both repeated measures and 
inter-group factors, you should make explicit your model on the 
intra-subject part of the data. You can't do Chi-square comparisons on 
count data that mix independent and dependent measures.

But you can define a Bernoulli logistic model at the individual single 
response level, and define a proper subject factor, and a stimulus-type 
factor, and possibly an item factor (nested within the stimulus-type 
category). This may be viewed as a Rasch model of measurement.

Within this model, coefficient estimates on the subject factor are 
measures of individual ability that can be a posteriori introduced in an 
ANOVA.

In some cases, you can model the intra-subject part of the data as a 
temporal model, assuming for instance that subjects are getting more 
efficient as time goes on.

HTH,

Yvonnick.