R help - Sep 2005 - fitting distribution tails

Hello,
 
I want to fit a distribution to a dataset. Important is not the
"overall"
fitting but the fitting in the tail (e.g. all observations > x or the n
highest values). Standard ML-estimation sometimes doesn't work here very
well. We see that especially when we have truncated datasets the algorithms
won't converge. In the case of lognormal distribution: It seems that the
farer the truncation point is away from the peak of the whole distribution
the more unlikely is the convergence.
 
So I think to do the following. And my questions are: Before I try to do it
with basic R-knowledge on my own ... maybe there is a similar solution
already available in R. And:  maybe somebody can give me further reading on
this topic or has other/better ideas how to cope this type of problem
(except EVT-Approaches).
 
First I produce the quantiles for all points of my dataset. I fix that the
fitting will be done for the n largest values. Then an optimization
algorithm starts. The objective function could be a goodness-of-fit
criterion, for a first try e.g: Minimize the sum over all squared deltas
[empirical - theoretical distribution]. In *any* case should be found
parameters that fulfill the condition. The criterion should be able to
overweight observations the higher they are.
 
What do you think about ?
 
Regards, Carsten

	[[alternative HTML version deleted]]

Not an R-response, but see this reference:

Dolan CV, van der Maas HLJ, Molenaar PCM
A framework for ML estimation of parameters of (mixtures of) common reaction
time distributions given optional truncation or censoring??
 BEHAVIOR RESEARCH METHODS INSTRUMENTS & COMPUTERS 34 (3): 304-323 AUG 2002

on estimating distribution parameters on truncated data sets, there is an
accompanying program available (which may or may not easily port to R ...)

hth, ingmar

> From: "Carsten Steinhoff" <carsten.steinhoff at
stud.uni-goettingen.de>
> Date: Wed, 7 Sep 2005 13:15:39 +0200
> To: <r-help at stat.math.ethz.ch>
> Subject: [R] fitting distribution tails
> 
> Hello,
> 
> I want to fit a distribution to a dataset. Important is not the
"overall"
> fitting but the fitting in the tail (e.g. all observations > x or the n
> highest values). Standard ML-estimation sometimes doesn't work here
very
> well. We see that especially when we have truncated datasets the algorithms
> won't converge. In the case of lognormal distribution: It seems that
the
> farer the truncation point is away from the peak of the whole distribution
> the more unlikely is the convergence.
> 
> So I think to do the following. And my questions are: Before I try to do it
> with basic R-knowledge on my own ... maybe there is a similar solution
> already available in R. And:  maybe somebody can give me further reading on
> this topic or has other/better ideas how to cope this type of problem
> (except EVT-Approaches).
> 
> First I produce the quantiles for all points of my dataset. I fix that the
> fitting will be done for the n largest values. Then an optimization
> algorithm starts. The objective function could be a goodness-of-fit
> criterion, for a first try e.g: Minimize the sum over all squared deltas
> [empirical - theoretical distribution]. In *any* case should be found
> parameters that fulfill the condition. The criterion should be able to
> overweight observations the higher they are.
> 
> What do you think about ?
> 
> Regards, Carsten
> 
> [[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide!
http://www.R-project.org/posting-guide.html
>