Spencer Graves
2005-Feb-02 19:28 UTC
[R] A modified log transformation with real finite values for negatives and zeros?
Does anyone have any ideas (or even experience) regarding a modified log transformation that would assign real finite values to zeros and negative numbers? I encounter this routinely in a couple of different situations: * Physical measurements that are often lognormally distributed except for values that are less than additive normal measurement error. I'd like to take logarithms of the clearly positive values and assign some smaller finite number(s) for values less than or equal to zero. I also might like to decompose the values into mean plus variance of the logs plus variance of additive normal noise. However, that would require more machinery than is appropriate for exploratory data analysis. * Integers most of which are plausibly Poisson counts but include a few negative values. People in manufacturing sometimes report the number of defects "added" between two steps in the process, computed as the difference between the number counted before and after intervening steps. These counts are occasionally negative either because defects are removed in processing or because of a miscount either before or after. For an example, see "www.prodsyse.com/log0". There, you can also download working R code for such a transformation along with PowerPoint slides documenting some of the logic behind the code. It's not included here, because it's too much for a standard R post. Comments? Thanks, spencer graves
roger koenker
2005-Feb-02 19:51 UTC
[R] A modified log transformation with real finite values for negatives and zeros?
Bickel and Doksum (JASA, 1981) discuss a modified version of the Box-Cox transformation that looks like this: y -> ( sgn(y)* abs(y)^lambda -1)/lambda and in the original Box-Cox paper there was an offset parameter that gives rise to some somewhat peculiar likelihood theory as in the 3-parameter log-normal where one gets an unbounded likelihood by letting the threshold parameter approach the first order statistic from below, but for which the likeihood equations seem to provide a perfectly sensible root. url: www.econ.uiuc.edu/~roger Roger Koenker email rkoenker at uiuc.edu Department of Economics vox: 217-333-4558 University of Illinois fax: 217-244-6678 Champaign, IL 61820 On Feb 2, 2005, at 1:28 PM, Spencer Graves wrote:> Does anyone have any ideas (or even experience) regarding a > modified log transformation that would assign real finite values to > zeros and negative numbers? I encounter this routinely in a couple of > different situations: > * Physical measurements that are often lognormally distributed > except for values that are less than additive normal measurement > error. I'd like to take logarithms of the clearly positive values and > assign some smaller finite number(s) for values less than or equal to > zero. I also might like to decompose the values into mean plus > variance of the logs plus variance of additive normal noise. However, > that would require more machinery than is appropriate for exploratory > data analysis. > * Integers most of which are plausibly Poisson counts but include > a few negative values. People in manufacturing sometimes report the > number of defects "added" between two steps in the process, computed > as the difference between the number counted before and after > intervening steps. These counts are occasionally negative either > because defects are removed in processing or because of a miscount > either before or after. > For an example, see "www.prodsyse.com/log0". There, you can also > download working R code for such a transformation along with > PowerPoint slides documenting some of the logic behind the code. It's > not included here, because it's too much for a standard R post. > Comments? Thanks, > spencer graves > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! > http://www.R-project.org/posting-guide.html