Dear list, could someone point me to the normal score transform and it's use in R? I would like to transform my data with normal score transform, do some geostatsitical predictions and would like to transform the estimated values back including the estimation variance. Any suggestions? Ulrich -- __________________________________________________ Ulrich Leopold MSc. Department of Physical Geography Institute for Biodiversity and Ecosystem Dynamics Faculty of Science University of Amsterdam Nieuwe Achtergracht 166 NL-1018WV Amsterdam Phone: +31-(0)20-525-7456 (7451 Secretary) Fax: +31-(0)20-525-7431 Email: uleopold at science.uva.nl http://www.frw.uva.nl/soil/Welcome.html Check us also out at: Netherlands Centre for Geo-ecological Research http://www.frw.uva.nl/icg -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
How would I do a backtransform of predicted values (derived from the given data) AND the variance? Ulrich Liaw, Andy wrote: > The "normal scores" are just ranks of the data mapped onto the normal CDF, > so something like: > > qnorm((rank(x) - 0.375)/(sum(!is.na(x)) + .25)) > > should do what you want. Another alternative (as calculated in qqnorm) is > > qnorm(ppoints(x)) > > Andy > > >>-----Original Message----- >>From: Ulrich Leopold [mailto:uleopold at science.uva.nl] >>Sent: Thursday, September 12, 2002 12:59 PM >>To: r-help at stat.math.ethz.ch >>Subject: [R] normal score transform >> >> >>Dear list, >> >>could someone point me to the normal score transform and it's >>use in R? >> >>I would like to transform my data with normal score >>transform, do some >>geostatsitical predictions and would like to transform the >>estimated values >>back including the estimation variance. >> >>Any suggestions? >> >>Ulrich >>-- >>__________________________________________________ >> >>Ulrich Leopold MSc. >> >>Department of Physical Geography >>Institute for Biodiversity and Ecosystem Dynamics >>Faculty of Science >>University of Amsterdam >>Nieuwe Achtergracht 166 >>NL-1018WV Amsterdam >> >>Phone: +31-(0)20-525-7456 (7451 Secretary) >>Fax: +31-(0)20-525-7431 >>Email: uleopold at science.uva.nl >>http://www.frw.uva.nl/soil/Welcome.html >> >>Check us also out at: >>Netherlands Centre for Geo-ecological Research >>http://www.frw.uva.nl/icg >> >> >> >>-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-. >>-.-.-.-.-.-.-.-.- >>r-help mailing list -- Read >>http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html >>Send "info", "help", or "[un]subscribe" >>(in the "body", not the subject !) To: >>r-help-request at stat.math.ethz.ch >>_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._. >>_._._._._._._._._ >> > > > > ------------------------------------------------------------------------------ > Notice: This e-mail message, together with any attachments, contains information of Merck & Co., Inc. (Whitehouse Station, New Jersey, USA) that may be confidential, proprietary copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by e-mail and then delete it. > > ============================================================================= > -- __________________________________________________ Ulrich Leopold MSc. Department of Physical Geography Institute for Biodiversity and Ecosystem Dynamics Faculty of Science University of Amsterdam Nieuwe Achtergracht 166 NL-1018WV Amsterdam Phone: +31-(0)20-525-7456 (7451 Secretary) Fax: +31-(0)20-525-7431 Email: uleopold at science.uva.nl http://www.frw.uva.nl/soil/Welcome.html Check us also out at: Netherlands Centre for Geo-ecological Research http://www.frw.uva.nl/icg -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
I suppose you can use approx() to linearly interpolate. For example:> x <- rnorm(10, mean=100, sd=10) > x[1] 92.46931 96.71141 103.82640 110.57520 88.30825 106.35602 90.60754 [8] 102.29406 94.19188 107.28770> z <- qnorm(ppoints(x)) > z[1] -1.5466353 -1.0004905 -0.6554235 -0.3754618 -0.1225808 0.1225808 [7] 0.3754618 0.6554235 1.0004905 1.5466353> z.test <- rnorm(10) ## pretend these are the "predicted values" > z.test[1] 1.1733989 1.3471282 -0.2409461 0.6951916 -0.6365362 -1.0181946 [7] -0.5347734 2.2335172 0.4747204 0.4917223> x.test <- approx(z, x, ztest)$y > x.test[1] 100.35171 98.74701 94.90596 107.39995 98.42442 94.84765 91.54209 [8] 106.87274 NA 103.25218 Note that approx() by default does not extrapolate (thus the NA), and the alternative (rule=2) probably isn't appropriate for this application. HTH, Andy> -----Original Message----- > From: Ulrich Leopold [mailto:uleopold at science.uva.nl] > Sent: Friday, September 13, 2002 4:10 AM > To: r-help at stat.math.ethz.ch > Subject: Re: [R] normal score transform > > > How would I do a backtransform of predicted values (derived > from the given > data) AND the variance? > > Ulrich > > Liaw, Andy wrote: > > The "normal scores" are just ranks of the data mapped onto > the normal CDF, > > so something like: > > > > qnorm((rank(x) - 0.375)/(sum(!is.na(x)) + .25)) > > > > should do what you want. Another alternative (as > calculated in qqnorm) is > > > > qnorm(ppoints(x)) > > > > Andy > > > > > >>-----Original Message----- > >>From: Ulrich Leopold [mailto:uleopold at science.uva.nl] > >>Sent: Thursday, September 12, 2002 12:59 PM > >>To: r-help at stat.math.ethz.ch > >>Subject: [R] normal score transform > >> > >> > >>Dear list, > >> > >>could someone point me to the normal score transform and it's > >>use in R? > >> > >>I would like to transform my data with normal score > >>transform, do some > >>geostatsitical predictions and would like to transform the > >>estimated values > >>back including the estimation variance. > >> > >>Any suggestions? > >> > >>Ulrich > >>-- > >>__________________________________________________ > >> > >>Ulrich Leopold MSc. > >> > >>Department of Physical Geography > >>Institute for Biodiversity and Ecosystem Dynamics > >>Faculty of Science > >>University of Amsterdam > >>Nieuwe Achtergracht 166 > >>NL-1018WV Amsterdam > >> > >>Phone: +31-(0)20-525-7456 (7451 Secretary) > >>Fax: +31-(0)20-525-7431 > >>Email: uleopold at science.uva.nl > >>http://www.frw.uva.nl/soil/Welcome.html > >> > >>Check us also out at: > >>Netherlands Centre for Geo-ecological Research > >>http://www.frw.uva.nl/icg > >> > >> > >> > >>-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-. > >>-.-.-.-.-.-.-.-.- > >>r-help mailing list -- Read > >>http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html > >>Send "info", "help", or "[un]subscribe" > >>(in the "body", not the subject !) To: > >>r-help-request at stat.math.ethz.ch > >>_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._. > >>_._._._._._._._._ > >> > > > > > > > > > -------------------------------------------------------------- > ---------------- > > Notice: This e-mail message, together with any > attachments, contains > information of Merck & Co., Inc. (Whitehouse Station, New > Jersey, USA) that > may be confidential, proprietary copyrighted and/or legally > privileged, and > is intended solely for the use of the individual or entity > named in this > message. If you are not the intended recipient, and have > received this > message in error, please immediately return this by e-mail > and then delete it. > > > > > =============================================================> ===============> > > > -- > __________________________________________________ > > Ulrich Leopold MSc. > > Department of Physical Geography > Institute for Biodiversity and Ecosystem Dynamics > Faculty of Science > University of Amsterdam > Nieuwe Achtergracht 166 > NL-1018WV Amsterdam > > Phone: +31-(0)20-525-7456 (7451 Secretary) > Fax: +31-(0)20-525-7431 > Email: uleopold at science.uva.nl > http://www.frw.uva.nl/soil/Welcome.html > > Check us also out at: > Netherlands Centre for Geo-ecological Research > http://www.frw.uva.nl/icg > > > > > -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-. > -.-.-.-.-.-.-.-.- > r-help mailing list -- Readhttp://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._. _._ ------------------------------------------------------------------------------ Notice: This e-mail message, together with any attachments, contains information of Merck & Co., Inc. (Whitehouse Station, New Jersey, USA) that may be confidential, proprietary copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named on this message. If you are not the intended recipient, and have received this message in error, please immediately return this by e-mail and then delete it. ============================================================================= -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
I don't know of a good way to do this. If the transformation has a nice form, then one can use the good old delta method (Taylor expansion). MASS has coverage and code, in the context of parameter estimation. However I don't think that can be applied to the normal scores transformation (at least I don't see how). Excuse me for being nosy, but why would you need to do this? Andy> -----Original Message----- > From: Ulrich Leopold [mailto:uleopold at science.uva.nl] > Sent: Friday, September 13, 2002 8:24 AM > To: r-help at stat.math.ethz.ch > Subject: Re: [R] normal score transform > > > But is it also possible to transform the prediction VARIANCE > back to non > normalized variances? I do not have these variances before > hand. Or would be > a solution to use the standard deviation as this will be > within the range of > the predictions? I'm not quite sure... > > Ulrich > > Liaw, Andy wrote: > > I suppose you can use approx() to linearly interpolate. > For example: > > > > > >>x <- rnorm(10, mean=100, sd=10) > >>x > > > > [1] 92.46931 96.71141 103.82640 110.57520 88.30825 > 106.35602 90.60754 > > [8] 102.29406 94.19188 107.28770 > > > >>z <- qnorm(ppoints(x)) > >>z > > > > [1] -1.5466353 -1.0004905 -0.6554235 -0.3754618 -0.1225808 > 0.1225808 > > [7] 0.3754618 0.6554235 1.0004905 1.5466353 > > > >>z.test <- rnorm(10) ## pretend these are the "predicted values" > >>z.test > > > > [1] 1.1733989 1.3471282 -0.2409461 0.6951916 -0.6365362 > -1.0181946 > > [7] -0.5347734 2.2335172 0.4747204 0.4917223 > > > >>x.test <- approx(z, x, ztest)$y > >>x.test > > > > [1] 100.35171 98.74701 94.90596 107.39995 98.42442 > 94.84765 91.54209 > > [8] 106.87274 NA 103.25218 > > > > Note that approx() by default does not extrapolate (thus > the NA), and the > > alternative (rule=2) probably isn't appropriate for this > application. > > > > HTH, > > Andy > > > > > >>-----Original Message----- > >>From: Ulrich Leopold [mailto:uleopold at science.uva.nl] > >>Sent: Friday, September 13, 2002 4:10 AM > >>To: r-help at stat.math.ethz.ch > >>Subject: Re: [R] normal score transform > >> > >> > >>How would I do a backtransform of predicted values (derived > >>from the given > >>data) AND the variance? > >> > >>Ulrich > >> > >>Liaw, Andy wrote: > >> > The "normal scores" are just ranks of the data mapped onto > >>the normal CDF, > >> > so something like: > >> > > >> > qnorm((rank(x) - 0.375)/(sum(!is.na(x)) + .25)) > >> > > >> > should do what you want. Another alternative (as > >>calculated in qqnorm) is > >> > > >> > qnorm(ppoints(x)) > >> > > >> > Andy > >> > > >> > > >> >>-----Original Message----- > >> >>From: Ulrich Leopold [mailto:uleopold at science.uva.nl] > >> >>Sent: Thursday, September 12, 2002 12:59 PM > >> >>To: r-help at stat.math.ethz.ch > >> >>Subject: [R] normal score transform > >> >> > >> >> > >> >>Dear list, > >> >> > >> >>could someone point me to the normal score transform and it's > >> >>use in R? > >> >> > >> >>I would like to transform my data with normal score > >> >>transform, do some > >> >>geostatsitical predictions and would like to transform the > >> >>estimated values > >> >>back including the estimation variance. > >> >> > >> >>Any suggestions? > >> >> > >> >>Ulrich > >> >>-- > >> >>__________________________________________________ > >> >> > >> >>Ulrich Leopold MSc. > >> >> > >> >>Department of Physical Geography > >> >>Institute for Biodiversity and Ecosystem Dynamics > >> >>Faculty of Science > >> >>University of Amsterdam > >> >>Nieuwe Achtergracht 166 > >> >>NL-1018WV Amsterdam > >> >> > >> >>Phone: +31-(0)20-525-7456 (7451 Secretary) > >> >>Fax: +31-(0)20-525-7431 > >> >>Email: uleopold at science.uva.nl > >> >>http://www.frw.uva.nl/soil/Welcome.html > >> >> > >> >>Check us also out at: > >> >>Netherlands Centre for Geo-ecological Research > >> >>http://www.frw.uva.nl/icg > >> >> > >> >> > >> >> > >> >>-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-. > >> >>-.-.-.-.-.-.-.-.- > >> >>r-help mailing list -- Read > >> >>http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html > >> >>Send "info", "help", or "[un]subscribe" > >> >>(in the "body", not the subject !) To: > >> >>r-help-request at stat.math.ethz.ch > >> >>_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._. > >> >>_._._._._._._._._ > >> >> > >> > > >> > > >> > > >> > > >>-------------------------------------------------------------- > >>---------------- > >> > Notice: This e-mail message, together with any > >>attachments, contains > >>information of Merck & Co., Inc. (Whitehouse Station, New > >>Jersey, USA) that > >>may be confidential, proprietary copyrighted and/or legally > >>privileged, and > >>is intended solely for the use of the individual or entity > >>named in this > >>message. If you are not the intended recipient, and have > >>received this > >>message in error, please immediately return this by e-mail > >>and then delete it. > >> > > >> > > >>=============================================================> >>===============> >> > > >> > >>-- > >>__________________________________________________ > >> > >>Ulrich Leopold MSc. > >> > >>Department of Physical Geography > >>Institute for Biodiversity and Ecosystem Dynamics > >>Faculty of Science > >>University of Amsterdam > >>Nieuwe Achtergracht 166 > >>NL-1018WV Amsterdam > >> > >>Phone: +31-(0)20-525-7456 (7451 Secretary) > >>Fax: +31-(0)20-525-7431 > >>Email: uleopold at science.uva.nl > >>http://www.frw.uva.nl/soil/Welcome.html > >> > >>Check us also out at: > >>Netherlands Centre for Geo-ecological Research > >>http://www.frw.uva.nl/icg > >> > >> > >> > >> > >>-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-. > >>-.-.-.-.-.-.-.-.- > >>r-help mailing list -- Read > > > > http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html > > Send "info", "help", or "[un]subscribe" > > (in the "body", not the subject !) To: > r-help-request at stat.math.ethz.ch > > > _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._. > _._._._._._._. > > _._ > > > > > -------------------------------------------------------------- > ---------------- > > Notice: This e-mail message, together with any attachments, > contains information of Merck & Co., Inc. (Whitehouse > Station, New Jersey, USA) that may be confidential, > proprietary copyrighted and/or legally privileged, and is > intended solely for the use of the individual or entity named > on this message. If you are not the intended recipient, and > have received this message in error, please immediately > return this by e-mail and then delete it. > > > > > =============================================================> ===============> > > > -- > __________________________________________________ > > Ulrich Leopold MSc. > > Department of Physical Geography > Institute for Biodiversity and Ecosystem Dynamics > Faculty of Science > University of Amsterdam > Nieuwe Achtergracht 166 > NL-1018WV Amsterdam > > Phone: +31-(0)20-525-7456 (7451 Secretary) > Fax: +31-(0)20-525-7431 > Email: uleopold at science.uva.nl > http://www.frw.uva.nl/soil/Welcome.html > > Check us also out at: > Netherlands Centre for Geo-ecological Research > http://www.frw.uva.nl/icg > > > > -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-. > -.-.-.-.-.-.-.-.- > r-help mailing list -- Readhttp://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._. _._ ------------------------------------------------------------------------------ Notice: This e-mail message, together with any attachments, contains information of Merck & Co., Inc. (Whitehouse Station, New Jersey, USA) that may be confidential, proprietary copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by e-mail and then delete it. ============================================================================= -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Dear Ulrich One way to back-transform is by simulation. Procedure 1. Make 1000 simulation from a normal distribution with mean = predicted mean, variance = prediction variance 2. back-transform the simulations 3. Calculate mean and variance of the simulations. For the normal-score-transform you need to define what you mean by the inverse of such transform [as a function taking any real number, not just one of your data values]. That sounds odd to me, so why not use another transformation ? Box-Cox transformations are implemented in package geoR [and if I remember correctly, back-transformation by simulations is also implemented] Cheers Ole> But is it also possible to transform the prediction VARIANCE back to non > normalized variances? I do not have these variances before hand. Or would be > a solution to use the standard deviation as this will be within the range of > the predictions? I'm not quite sure...-- Ole F. Christensen Department of Mathematics and Statistics Fylde College, Lancaster University Lancaster, LA1 4YF, England -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._