R help - Feb 2005 - Bayesian stepwise (was: Forward Stepwise regression based onpartial F test)

oops,

Forgot to cc to the list.

Regards,

Mike

-----Original Message-----
From: dr mike [mailto:dr.mike at ntlworld.com] 
Sent: 24 February 2005 19:21
To: 'Spencer Graves'
Subject: RE: [R] Bayesian stepwise (was: Forward Stepwise regression based
onpartial F test)

Spencer,

Obviously the problem is one of supersaturation. In view of that, are you
aware of the following?

A Two-Stage Bayesian Model Selection Strategy for Supersaturated Designs
Authors: Beattie S. D; Fong D. K. H; Lin D. K. J
Source: Technometrics, 1 February 2002, vol. 44, no. 1, pp. 55-63 

And:

Analysis Methods for Supersaturated Design: Some Comparisons
Authors: Li R; Lin D. K. J
Source: Journal of Data Sciences, 1, 2003, pp. 249-260

The latter is available for download in full (pdf) by googling for the
title.

HTH

Mike

-----Original Message-----
From: r-help-bounces at stat.math.ethz.ch
[mailto:r-help-bounces at stat.math.ethz.ch] On Behalf Of Spencer Graves
Sent: 24 February 2005 17:29
To: Frank E Harrell Jr
Cc: r-help at stat.math.ethz.ch
Subject: [R] Bayesian stepwise (was: Forward Stepwise regression based
onpartial F test)

      Does anyone know of research fully Bayesian stepwise procedures
assuming that models not considered by the stepwise would essentially have
zero posterior probability? 

      I need to analyze the results of ad hoc experiments run in
manufacturing with crazy confounding and possible supersaturation (i.e.,
more potentially explanatory variables than runs), when each run is very 
expensive in both time and money.   There have to be ways to summarize 
concisely and intelligently what the data can tell us and what remains
uncertain, including the level of partial confounding between alternative
explanations.  I think I've gotten reasonable results with my own
modification of Venables & Ripley's stepAIC to compute an approximate
posterior over tested models using the AICc criterion described, e.g., by
Burnham and Anderson (2002) Model Selection and Multi-Model Inference
(Springer).  Preliminary simulations showed that when I used the naive prior
(that all models are equally likely, including the null model), the null
model is usually rejected when true.  What a surprise!  I think I can fix
that using a more intelligent prior.  I also think I can evaluate the
partial confounding between alternative models by studying the correlation
matrix between the predictions of alternative models. 

      Comments?
      Thanks,
      Spencer Graves

Frank E Harrell Jr wrote:
> Smit, Robin wrote:
>
>> I am hoping to get some advise on the following:
>>  
>> I am looking for an automatic variable selection procedure to reduce 
>> the number of potential predictor variables (~ 50) in a multiple 
>> regression model.
>>  
>> I would be interested to use the forward stepwise regression using 
>> the partial F test. I have looked into possible R-functions but could 
>> not find this particular approach.
>> There is a function (stepAIC) that uses the Akaike criterion or 
>> Mallow's Cp criterion. In addition, the drop1 and add1 functions
came
>> closest to what I want but with them I cannot perform the required 
>> procedure. Do you have any ideas?
>> Kind regards,
>> Robin Smit
>> --------------------------------------------
>> Business Unit TNO Automotive
>> Environmental Studies & Testing
>> PO Box 6033, 2600 JA Delft
>> THE NETHERLANDS
>
>
> Robin,
>
> If you are looking for a method that does not offer the best 
> predictive accuracy and that violates every aspect of statistical 
> inference, you are on the right track.  See 
> http://www.stata.com/support/faqs/stat/stepwise.html for details.
>
______________________________________________
R-help at stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide!
http://www.R-project.org/posting-guide.html