R help - Jan 2005 - Kolmogorov-Smirnof test for lognormal distribution with estimated parameters

Hello all,

Would somebody be kind enough to show me how to do a KS test in R for a
lognormal distribution with ESTIMATED parameters. The R function
ks.test()says "the parameters specified must be prespecified and not
estimated from the data" Is there a way to correct this when one uses
estimated data?

Regards,

Kwabena.

--------------------------------------------
Kwabena Adusei-Poku
University of Goettingen
Institute of Statistics and Econometrics
Platz der Goettingen Sieben 5
37073 Goettingen
Germany
Tel: +49-(0)551-394794

Hi,

I believe that to performe KS test parameters must not
be estimated by sample data.

Despite some advantages, the KS test has several
important limitations:

   1. It only applies to continuous distributions.
   2. It tends to be more sensitive near the center of
the distribution than at the tails.
-->3. Perhaps the most serious limitation is that the
distribution must be fully specified. That is, if
location, scale, and shape parameters are estimated
from the data, the critical region of the K-S test is
no longer valid. It typically must be determined by
simulation. <--

Due to limitations 2 and 3 above, many analysts prefer
to use the Anderson-Darling goodness-of-fit test.
However, the Anderson-Darling test is only available
for a few specific distributions. 

See:

http://www.itl.nist.gov/div898/handbook/eda/section3/eda35g.htm

for KS test

http://www.itl.nist.gov/div898/handbook/eda/section3/eda35e.htm
for Anderson-Darling test

http://www.itl.nist.gov/div898/handbook/eda/section3/eda35f.htm
X2 test

I suggest you to use th chisquare test.

Hoping to help you.
Best,
Vito

You wrote:

Hello all,

Would somebody be kind enough to show me how to do a
KS test in R for a lognormal distribution with
ESTIMATED parameters. The R function ks.test()says
"the parameters specified must be prespecified and not
estimated from the data" Is there a way to correct
this when one uses
estimated data?

Regards,

Kwabena.

====Diventare costruttori di soluzioni
Became solutions' constructors

"The business of the statistician is to catalyze 
the scientific learning process."  
George E. P. Box

Top 10 reasons to become a Statistician

     1. Deviation is considered normal
     2. We feel complete and sufficient
     3. We are 'mean' lovers
     4. Statisticians do it discretely and continuously
     5. We are right 95% of the time
     6. We can legally comment on someone's posterior distribution
     7. We may not be normal, but we are transformable
     8. We never have to say we are certain
     9. We are honestly significantly different
    10. No one wants our jobs


Visitate il portale http://www.modugno.it/
e in particolare la sezione su Palese  http://www.modugno.it/archivio/palese/

Hi Kwabena,

in addition to my preview reply:

you can use some normality test included in nortest
package transfroming log-normal data in normal data
using log-transformation.

See:
http://cran.r-mirror.de/src/contrib/Descriptions/nortest.html

ad.test  Anderson-Darling test for normality
cvm.test Cramer-von Mises test for normality
lillie.test Lilliefors (Kolmogorov-Smirnov) test for  
                     normality 
Pearson.test Pearson chi-square test for normality
sf.test    Shapiro-Francia test for normality

Best Regards,

Vito


you wrote:

Hello all,

Would somebody be kind enough to show me how to do a
KS test in R for a
lognormal distribution with ESTIMATED parameters. The
R function
ks.test()says "the parameters specified must be
prespecified and not
estimated from the data" Is there a way to correct
this when one uses
estimated data?

Regards,

Kwabena.

--------------------------------------------
Kwabena Adusei-Poku
University of Goettingen
Institute of Statistics and Econometrics
Platz der Goettingen Sieben 5
37073 Goettingen
Germany
Tel: +49-(0)551-394794

====Diventare costruttori di soluzioni
Became solutions' constructors

"The business of the statistician is to catalyze 
the scientific learning process."  
George E. P. Box

Top 10 reasons to become a Statistician

     1. Deviation is considered normal
     2. We feel complete and sufficient
     3. We are 'mean' lovers
     4. Statisticians do it discretely and continuously
     5. We are right 95% of the time
     6. We can legally comment on someone's posterior distribution
     7. We may not be normal, but we are transformable
     8. We never have to say we are certain
     9. We are honestly significantly different
    10. No one wants our jobs


Visitate il portale http://www.modugno.it/
e in particolare la sezione su Palese  http://www.modugno.it/archivio/palese/

Hi Kwabena

I did once a simulation, generating normal distributed values
(500 values) and calculating a KS test with estimated
parameters. For 10000 times repeating this test I got about
1 significant tests (on a level alpha=0.05 I'm expecting about 500 
significant tests by chance)
So I think if you estiamte the parameters from the data, you fit
to good and the used distribution of the test statistic is not
adequate as it is indicated in the help page you cited. There
(in the help page) is some literature, but it is no easy stuff
to read.
Furthermore I know no implementation of an KS test which
accounts for this estimation of the parameter.

I recommend a graphical tool instead of a test:

x <- rlnorm(100)
qqnorm(log(x))

See also ?qqnorm and ?qqplot.

If you insist on testing a theoretical distribution be aware
that a non significant test does not mean that your data has the
tested distribution (especially if you have few data, there is
no power in the test to detect deviations from the theoretical
distribution and the conclusion that the data fits well is
trappy)

If there are enough data I'd prefer a chi square test to the KS
test (but even there I use graphical tools instead). 

See ?chisq

For this test you have to specify classes and this is 
subjective (you can't avoid this).

You can reduce the DF of the expected chi square distribution
(under H_0) by the number of estimated parameters from the data
and will get better results. 

DF = number of classes - 1 - estimated parameters

I think this test is more powerful than the KS test,
particularly if you must estimate the parameters from data.

Regards,

Christoph

-- 
Christoph Buser <buser at stat.math.ethz.ch>
Seminar fuer Statistik, LEO C11
ETH (Federal Inst. Technology)	8092 Zurich	 SWITZERLAND
phone: x-41-1-632-5414		fax: 632-1228
http://stat.ethz.ch/~buser/



Kwabena Adusei-Poku writes:
 > Hello all,
 > 
 > Would somebody be kind enough to show me how to do a KS test in R for a
 > lognormal distribution with ESTIMATED parameters. The R function
 > ks.test()says "the parameters specified must be prespecified and not
 > estimated from the data" Is there a way to correct this when one uses
 > estimated data?
 > 
 > Regards,
 > 
 > Kwabena.
 > 
 > --------------------------------------------
 > Kwabena Adusei-Poku
 > University of Goettingen
 > Institute of Statistics and Econometrics
 > Platz der Goettingen Sieben 5
 > 37073 Goettingen
 > Germany
 > Tel: +49-(0)551-394794
 > 
 > ______________________________________________
 > R-help at stat.math.ethz.ch mailing list
 > https://stat.ethz.ch/mailman/listinfo/r-help
 > PLEASE do read the posting guide!
http://www.R-project.org/posting-guide.html