R help - Nov 2007 - Kolmogorov-Smirnoff test

I am trying to determine whether two samples are identical or not. I'm 
aware that somebody can use the Kolmogorov-Smirnoff test to compare 
empirical distributions, but since my samples have ties I'm not sure if 
I'm getting the right p-values for the comparison. Can the 
Kolmogorov-Smirnoff test be adjusted for the case when ties exists and 
are there any functions that already exists in R ( Kolmogorov-Smirnoff 
test )performing  that can be used in the case of the existance of ties?

Thank you in advance for your help.

Oarabile

On 06-Nov-07 15:53:53, Oarabile Molaodi wrote:
> I am trying to determine whether two samples are identical or not.
> I'm aware that somebody can use the Kolmogorov-Smirnoff test to
> compare empirical distributions, but since my samples have ties
> I'm not sure if I'm getting the right p-values for the comparison.
> Can the Kolmogorov-Smirnoff test be adjusted for the case when ties
> exists and are there any functions that already exists in R
> (Kolmogorov-Smirnoff test )performing  that can be used in the case
> of the existance of ties?
> 
> Thank you in advance for your help.
> 
> Oarabile
Tests like the Kolmogorov-Smirnov whose theoretical null
distribution assume continuous random variables (hence
wothout ties) do not have definite null distributions
when ties are possible. Whatever null distribution the
test may have when ties are present (e.g. due to data
being recorded to a relatively coarse precision) will
depend on the pattern of ties.

However, it is possible to investigate the effect of
ties on the P-value by randomly breaking ties.

For instance, suppose your data are recorded to a precision
of 0.1, and you have two such samples X and Y, then let

X.rand <- X + 0.0001*runif(length(X)
Y.rand <- Y + 0.0001*runif(length(Y)

and then do a K-S test on X.rand vs Y.rand.

You will get a P-value. Repeat this many time. You will get
a distribution of P-values. You can extract any relevant
property of this distrobution of P-values, for instance
its mean, it's 95th percentile (so you can be 96% confident
that the tie-broken P-value is less than this value).
and so on.

Hoping this helps,
Ted.

--------------------------------------------------------------------
E-Mail: (Ted Harding) <Ted.Harding at manchester.ac.uk>
Fax-to-email: +44 (0)870 094 0861
Date: 06-Nov-07                                       Time: 16:23:34
------------------------------ XFMail ------------------------------

A bootstrap Kolmogorov-Smirnoff test will have the correct test level
even if there are ties---i.e., even if non-continuous distributions
are being compared.  See Abadie, Alberto. 2002. ``Bootstrap Tests for
Distributional Treatment Effects in Instrumental Variable Models.'' 
Journal of the American Statistical Association, 97:457 (March)
284-292.

The algorithm is implemented in the ks.boot function of the Matching
package: http://sekhon.berkeley.edu/matching/ks.boot.html

Cheers,
Jas.

======================================Jasjeet S. Sekhon

Associate Professor
Travers Department of Political Science
Survey Research Center
UC Berkeley

http://sekhon.berkeley.edu/
V: 510-642-9974  F: 617-507-5524
======================================

Oarabile Molaodi writes:
 > I am trying to determine whether two samples are identical or not. I'm
 > aware that somebody can use the Kolmogorov-Smirnoff test to compare 
 > empirical distributions, but since my samples have ties I'm not sure
if
 > I'm getting the right p-values for the comparison. Can the 
 > Kolmogorov-Smirnoff test be adjusted for the case when ties exists and 
 > are there any functions that already exists in R ( Kolmogorov-Smirnoff 
 > test )performing  that can be used in the case of the existance of ties?
 > 
 > Thank you in advance for your help.
 > 
 > Oarabile
 > 
 >