Hello, Does anyone know of an R version of loess that allows more than 4 predictors and/or allows the specification of offsets? For that matter, does anyone know of _any_ version of loess that does either of the things I mention? Thanks, Paul Louisell 650-833-6254 ploua@allstate.com Research Associate (Statistician) Modeling & Data Analytics ARPC [[alternative HTML version deleted]]
On Mon, 22 Jan 2007, Louisell, Paul wrote:> Hello, > > Does anyone know of an R version of loess that allows more than 4 > predictors and/or allows the specification of offsets? For that matter, > does anyone know of _any_ version of loess that does either of the > things I mention?Why would you want offsets in a regression?: just subtract them from the lhs. (R's lm has gained offsets by analogy with glm, but the S original did not have them). If you would be more comfortable working with them, it would be very easy to create a modified version that supports them. Also, have you heard of the 'curse of dimensionality'? Localization even to 4 dimensions is no longer really an appropriate term, and Euclidean distance will be the main determinant of 'local' and is quite arbitrary.> Thanks, > > Paul Louisell > 650-833-6254 > ploua at allstate.com > Research Associate (Statistician) > Modeling & Data Analytics > ARPC > > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >-- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595
In response to your questions: I asked about including the offset for convenience; I currently put the offset in by subtracting it from the response, just as you suggest. The reason for including them is that I'm doing something slightly unusual with loess: I'm fitting loess to log((response+1)/offset) because the response is actually a vector of counts. This is intended to give a rough approximation to a Poisson regression; the reason for using loess is that the mean response should be approximated by a Poisson process with 4 predictor variables which can be divided into 2 pairs, each pair of which are geographic location coordinates. The two location pairs are expected to exhibit strong interaction; hence, the reason for fitting loess to all 4 predictors. I'm aware of the curse of dimensionality, but I have a very large dataset--over 600,000 observations. Since each pair of predictors represents a point on a grid, I think Euclidean distance is probably a good choice. And this brings me to the motivation for wanting to fit with 5 predictors: The offset is not _really_ an offset; it's just an approximation to what the real offset should be. Hence, I'd rather include it as a predictor than artificially force it to be included linearly with a coefficient of 1. I'm less concerned with linearity than I am with forcing the coefficient. In fact, I'd like to specify that it be unconditionally linear, but with an estimated coefficient. Thanks, Paul Louisell 650-833-6254 ploua at allstate.com Research Associate (Statistician) Modeling & Data Analytics ARPC -----Original Message----- From: Prof Brian Ripley [mailto:ripley at stats.ox.ac.uk] Sent: Monday, January 22, 2007 11:01 PM To: Louisell, Paul Cc: r-help at stat.math.ethz.ch Subject: Re: [R] Loess with more than 4 predictors / offsets On Mon, 22 Jan 2007, Louisell, Paul wrote:> Hello, > > Does anyone know of an R version of loess that allows more than 4 > predictors and/or allows the specification of offsets? For thatmatter,> does anyone know of _any_ version of loess that does either of the > things I mention?Why would you want offsets in a regression?: just subtract them from the lhs. (R's lm has gained offsets by analogy with glm, but the S original did not have them). If you would be more comfortable working with them, it would be very easy to create a modified version that supports them. Also, have you heard of the 'curse of dimensionality'? Localization even to 4 dimensions is no longer really an appropriate term, and Euclidean distance will be the main determinant of 'local' and is quite arbitrary.> Thanks, > > Paul Louisell > 650-833-6254 > ploua at allstate.com > Research Associate (Statistician) > Modeling & Data Analytics > ARPC > > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guidehttp://www.R-project.org/posting-guide.html> and provide commented, minimal, self-contained, reproducible code. >-- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595
I apologize for clogging up inboxes, but I realized I needed to amend what I said in my last comment below: In fact, I'd like to specify that it be unconditionally linear, but with estimated coefficients, _both an intercept and a slope_. If the "offset" were only multiplied by a nonzero constant c, this would have the effect of moving the whole response surface -log(c) units parallel to the response axis in the scenario I outline below. This would effectively give me the same thing I already have. Paul Louisell 650-833-6254 ploua at allstate.com Research Associate (Statistician) Modeling & Data Analytics ARPC -----Original Message----- From: Louisell, Paul Sent: Tuesday, January 23, 2007 12:40 PM To: 'Prof Brian Ripley' Cc: r-help at stat.math.ethz.ch Subject: RE: [R] Loess with more than 4 predictors / offsets In response to your questions: I asked about including the offset for convenience; I currently put the offset in by subtracting it from the response, just as you suggest. The reason for including them is that I'm doing something slightly unusual with loess: I'm fitting loess to log((response+1)/offset) because the response is actually a vector of counts. This is intended to give a rough approximation to a Poisson regression; the reason for using loess is that the mean response should be approximated by a Poisson process with 4 predictor variables which can be divided into 2 pairs, each pair of which are geographic location coordinates. The two location pairs are expected to exhibit strong interaction; hence, the reason for fitting loess to all 4 predictors. I'm aware of the curse of dimensionality, but I have a very large dataset--over 600,000 observations. Since each pair of predictors represents a point on a grid, I think Euclidean distance is probably a good choice. And this brings me to the motivation for wanting to fit with 5 predictors: The offset is not _really_ an offset; it's just an approximation to what the real offset should be. Hence, I'd rather include it as a predictor than artificially force it to be included linearly with a coefficient of 1. I'm less concerned with linearity than I am with forcing the coefficient. In fact, I'd like to specify that it be unconditionally linear, but with an estimated coefficient. Thanks, Paul Louisell 650-833-6254 ploua at allstate.com Research Associate (Statistician) Modeling & Data Analytics ARPC -----Original Message----- From: Prof Brian Ripley [mailto:ripley at stats.ox.ac.uk] Sent: Monday, January 22, 2007 11:01 PM To: Louisell, Paul Cc: r-help at stat.math.ethz.ch Subject: Re: [R] Loess with more than 4 predictors / offsets On Mon, 22 Jan 2007, Louisell, Paul wrote:> Hello, > > Does anyone know of an R version of loess that allows more than 4 > predictors and/or allows the specification of offsets? For thatmatter,> does anyone know of _any_ version of loess that does either of the > things I mention?Why would you want offsets in a regression?: just subtract them from the lhs. (R's lm has gained offsets by analogy with glm, but the S original did not have them). If you would be more comfortable working with them, it would be very easy to create a modified version that supports them. Also, have you heard of the 'curse of dimensionality'? Localization even to 4 dimensions is no longer really an appropriate term, and Euclidean distance will be the main determinant of 'local' and is quite arbitrary.> Thanks, > > Paul Louisell > 650-833-6254 > ploua at allstate.com > Research Associate (Statistician) > Modeling & Data Analytics > ARPC > > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guidehttp://www.R-project.org/posting-guide.html> and provide commented, minimal, self-contained, reproducible code. >-- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595
Possibly Parallel Threads
- R on UNIX Sun-Solaris 10.0 vs. S-Plus
- 64-bit R build with Studio 12 on Sparc v9
- 'asymmetric span' for 2D loess?
- Is there an implementation loess with more than 4 parametric predictors or a trick to similar effect?
- Studio 11 compiling of Rmath standalone versus compiling R