R help - Nov 2007 - AIC and model selection; not a R question

Hi,
 
I was wondering if someone could help me answer a question that is bound to come
up in my Master's defense. I'm using AIC to select models and my
question is how do I know that the models I developed a priori contain the
'best' models in the system. How do I not know that some models which I
didn't include aren't actually the 'best' model??
 
Thanks so much
Lynnette
 
~
Lynnette Dagenais
M.Sc. candidate
Department of Renewable Resources
University of Alberta, Edmonton
Canada


      Looking for a X-Mas gift?  Everybody needs a Flickr Pro Account.

 

http://www.flickr.com/gift/

	[[alternative HTML version deleted]]

On Sat, 24 Nov 2007, Lynnette Dagenais wrote:
> Hi,
>
> I was wondering if someone could help me answer a question that is bound 
> to come up in my Master's defense.
Lynnette,

I expect you will get some opinions here, but you might get a better 
answer from your committee.

Have you already asked the members of your committee?

If not, it would seem best to start there. In the institutions in which I 
have studied and/or taught, the role of the committee member was to 
educate as well as to test the candidate. Surely, committee members will 
understand the details of your research better than we can. There are many 
issues regarding models selection, and knowing how best to address them 
does depend on the details of what you wish to accomplish that they will 
know and we do not.

If you have asked, why are you asking here? Is there some difficulty or 
conflict that they have posed without providing the resolution? Are some 
members skeptical of model selection in general or of the particular 
procedure you use? Are there procedures that you have not used that 
some committee members would have favored?

HTH,

Chuck


I'm using AIC to select models and my 
> question is how do I know that the models I developed a priori contain 
> the 'best' models in the system. How do I not know that some models
> which I didn't include aren't actually the 'best' model??
>
> Thanks so much
> Lynnette
>
> ~
> Lynnette Dagenais
> M.Sc. candidate
> Department of Renewable Resources
> University of Alberta, Edmonton
> Canada
>
>
>      Looking for a X-Mas gift?  Everybody needs a Flickr Pro Account.
>
>
>
> http://www.flickr.com/gift/
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
Charles C. Berry                            (858) 534-2098
                                             Dept of Family/Preventive Medicine
E mailto:cberry at tajo.ucsd.edu	            UC San Diego
http://famprevmed.ucsd.edu/faculty/cberry/  La Jolla, San Diego 92093-0901

Lynnette Dagenais wrote:
> 
> Hi,
>  
> I was wondering if someone could help me answer a question that is bound
> to come up in my Master's defense. I'm using AIC to select models
and my
> question is how do I know that the models I developed a priori contain the
> 'best' models in the system. How do I not know that some models
which I
> didn't include aren't actually the 'best' model??
>  
> 
Answer: you can't (or probably not).

While there are some goodness-of-fit tests against generic alternatives,
most model selection approaches 
test only among a set of specified alternatives.

  On the other hand, if you have discrete data and a standard sampling
distribution
(binomial, Poisson, etc.), and the residuals from your most complex model
are _not_
significantly overdispersed, then you have some evidence that everything not
explained by your most complex model is due to (irreducible) sampling
variation.
(If you do have overdispersion that doesn't mean that your model is a bad
fit,
though.  Most ecological data, at least, is overdispersed.)

  good luck,
    Ben Bolker

-- 
View this message in context:
http://www.nabble.com/AIC-and-model-selection--not-a-R-question-tf4867273.html#a13929620
Sent from the R help mailing list archive at Nabble.com.