Dear All, A simple question: packages like fitdistr should be ideal to analyze samples of data taken from a univariate distribution, but what if rather than the raw data of the observations you are given directly and only a histogram? I was thinking about generating artificially a set of data corresponding to the counts binned in the histogram, but this sounds too cumbersome. Another question is the following: fitdistr provides the value of the log-likely hood function, but what if I want e.g. a chi square test to get some insight on the goodness of the fitting? I am sure there must be a way to get it straightforwardly without coding it myself. Many thanks Lorenzo
Prof Brian Ripley
2006-Jun-12 11:55 UTC
[R] Fitting Distributions Directly From a Histogram
On Mon, 12 Jun 2006, Lorenzo Isella wrote:> Dear All, > > A simple question: packages like fitdistr should be ideal to analyze > samples of data taken from a univariate distribution, but what if > rather than the raw data of the observations you are given directly > and only a histogram?(It is unusual for the actual data not to be available in real problems.)> I was thinking about generating artificially a set of data > corresponding to the counts binned in the histogram, but this sounds > too cumbersome.And that is not in any case a correct approach, since the mle from binned data is not the same as the mle from any dataset corresponding to the binned counts. You could use mle (package stats4) to estimate parameters from the likelihood of the grouped data.> Another question is the following: fitdistr provides the value of the > log-likely hood function, but what if I want e.g. a chi square test to > get some insight on the goodness of the fitting? > I am sure there must be a way to get it straightforwardly without > coding it myself.It is not well-defined, as you need to choose a binning to do a chisq test. Once you have (e.g. via cut and table), chisq.test() will produce the test statistic and refer it to a reference distribution (which will not take any parameter estimation into account). -- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595
Le Lundi 12 Juin 2006 06:51, Lorenzo Isella a ?crit?:> Dear All, > > A simple question: packages like fitdistr should be ideal to analyze > samples of data taken from a univariate distribution, but what if > rather than the raw data of the observations you are given directly > and only a histogram?Let's assume that you have not only the histogram itself, but also the breaks and the counts per bin. Then you have what grouped data --- at least that's how we call those in Actuarial Science. Maximum likelyhood estimation is feasible for such data, but it is slightly more complicated. "Loss Models" by Klugman, Panjer & Willmot (Wiley) covers this. I'm now thinking of adding this to my actuarial science package "actuar"...> I was thinking about generating artificially a set of data > corresponding to the counts binned in the histogram, but this sounds > too cumbersome. > Another question is the following: fitdistr provides the value of the > log-likely hood function, but what if I want e.g. a chi square test to > get some insight on the goodness of the fitting?Goodness of fit tests for grouped data are also covered in Loss Models.> I am sure there must be a way to get it straightforwardly without > coding it myself.Once you have the theory, I'm afraid for now you will have to code the estimation procedure yourself. Cheers.> Many thanks > > Lorenzo > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! > http://www.R-project.org/posting-guide.html-- Vincent Goulet, Professeur agr?g? ?cole d'actuariat Universit? Laval, Qu?bec Vincent.Goulet at act.ulaval.ca http://vgoulet.act.ulaval.ca