R help - Dec 2006 - RuleFit & quantreg: partial dependence plots; showing an effect

Dear List,

I would greatly appreciate help on the following matter:

The RuleFit program of Professor Friedman uses partial dependence plots
to explore the effect of an explanatory variable on the response
variable, after accounting for the average effects of the other
variables.  The plot method [plot(summary(rq(y ~ x1 + x2,
t=seq(.1,.9,.05))))] of Professor Koenker's quantreg program appears to
do the same thing.


Question:
Is there a difference between these two types of plot in the manner in which
they depict the relationship between explanatory variables and the response
variable ?

Thank you inav for your help.

Regards,
Mark Difford.

------------------------------------------------------------- 
Mark Difford
Ph.D. candidate, Botany Department,
Nelson Mandela Metropolitan University,
Port Elizabeth, SA.

They are entirely different:  Rulefit is a fiendishly clever  
combination of decision tree  formulation
of models and L1-regularization intended to select parsimonious fits  
to very complicated
responses yielding e.g. piecewise constant functions.  Rulefit   
estimates the  conditional
mean of the response over the covariate space, but permits a very  
flexible, but linear in
parameters specifications of the covariate effects on the conditional  
mean.  The quantile
regression plotting you refer to adopts a fixed, linear specification  
for conditional quantile
functions and given that specification depicts how the covariates  
influence the various
conditional quantiles of the response.   Thus, roughly speaking,  
Rulefit is focused on
flexibility in the x-space, maintaining the classical conditional  
mean objective; while
QR is trying to be more flexible in the y-direction, and maintaining  
a fixed, linear
in parameters specification for the covariate effects at each quantile.


url:    www.econ.uiuc.edu/~roger            Roger Koenker
email    rkoenker at uiuc.edu            Department of Economics
vox:     217-333-4558                University of Illinois
fax:       217-244-6678                Champaign, IL 61820


On Dec 20, 2006, at 4:17 AM, Mark Difford wrote:
> Dear List,
>
> I would greatly appreciate help on the following matter:
>
> The RuleFit program of Professor Friedman uses partial dependence  
> plots
> to explore the effect of an explanatory variable on the response
> variable, after accounting for the average effects of the other
> variables.  The plot method [plot(summary(rq(y ~ x1 + x2,
> t=seq(.1,.9,.05))))] of Professor Koenker's quantreg program  
> appears to
> do the same thing.
>
>
> Question:
> Is there a difference between these two types of plot in the manner  
> in which they depict the relationship between explanatory variables  
> and the response variable ?
>
> Thank you inav for your help.
>
> Regards,
> Mark Difford.
>
> -------------------------------------------------------------
> Mark Difford
> Ph.D. candidate, Botany Department,
> Nelson Mandela Metropolitan University,
> Port Elizabeth, SA.
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting- 
> guide.html
> and provide commented, minimal, self-contained, reproducible code.

Dear Professors Koenker and Varadhan,

Thank you for your detailed and engaging replies.  The (very) muddy waters clear
slowly, but only if I keep moving my hands!

Kind regards,
Mark Difford.
 
Mark Difford
Ph.D. candidate, Botany Department,
Nelson Mandela Metropolitan University,
Port Elizabeth, SA.

----- Original Message ----
From: roger koenker <rkoenker at uiuc.edu>
To: Mark Difford <mark_difford at yahoo.co.uk>
Cc: R-help list <r-help at stat.math.ethz.ch>
Sent: Wednesday, 20 December, 2006 3:57:02 PM
Subject: Re: [R] RuleFit & quantreg: partial dependence plots; showing an
effect

They are entirely different:  Rulefit is a fiendishly clever  
combination of decision tree  formulation
of models and L1-regularization intended to select parsimonious fits  
to very complicated
responses yielding e.g. piecewise constant functions.  Rulefit   
estimates the  conditional
mean of the response over the covariate space, but permits a very  
flexible, but linear in
parameters specifications of the covariate effects on the conditional  
mean.  The quantile
regression plotting you refer to adopts a fixed, linear specification  
for conditional quantile
functions and given that specification depicts how the covariates  
influence the various
conditional quantiles of the response.   Thus, roughly speaking,  
Rulefit is focused on
flexibility in the x-space, maintaining the classical conditional  
mean objective; while
QR is trying to be more flexible in the y-direction, and maintaining  
a fixed, linear
in parameters specification for the covariate effects at each quantile.


url:    www.econ.uiuc.edu/~roger            Roger Koenker
email    rkoenker at uiuc.edu            Department of Economics
vox:     217-333-4558                University of Illinois
fax:       217-244-6678                Champaign, IL 61820


On Dec 20, 2006, at 4:17 AM, Mark Difford wrote:
> Dear List,
>
> I would greatly appreciate help on the following matter:
>
> The RuleFit program of Professor Friedman uses partial dependence  
> plots
> to explore the effect of an explanatory variable on the response
> variable, after accounting for the average effects of the other
> variables.  The plot method [plot(summary(rq(y ~ x1 + x2,
> t=seq(.1,.9,.05))))] of Professor Koenker's quantreg program  
> appears to
> do the same thing.
>
>
> Question:
> Is there a difference between these two types of plot in the manner  
> in which they depict the relationship between explanatory variables  
> and the response variable ?
>
> Thank you inav for your help.
>
> Regards,
> Mark Difford.
>
> -------------------------------------------------------------
> Mark Difford
> Ph.D. candidate, Botany Department,
> Nelson Mandela Metropolitan University,
> Port Elizabeth, SA.
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting- 
> guide.html
> and provide commented, minimal, self-contained, reproducible code.




Send instant messages to your online friends http://uk.messenger.yahoo.com