Hello, Does logistic regression really provide better results than lda or qda ? (my purpose is not classification but highlighting of discriminant variables). If this is true, where could I get an R script performing stepwise logistic regression ? Thanks -- Daniel AMORESE Lab. M2C "Morphodynamique Continentale et C?ti?re" UMR CNRS 6143 Caen/Rouen Centre de G?omorphologie UCBN (Universit? de Caen Basse-Normandie) 14032 Caen Cedex FRANCE Ph: +33 (0)2 31 56 57 19 Fax: +33 (0)2 31 56 57 57 Courriel: amorese at geos.unicaen.fr geos.unicaen.fr/perso/da/daperso.html -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
At 06:25 PM 5/23/2002 +0200, Daniel Amor?se wrote:>Hello, >Does logistic regression really provide better results than lda or qda ? >(my purpose is not classification but highlighting of discriminant >variables)You've asked multiple questions about "stepwise quadratic regression" and now "stepwise logistic regression". You haven't really explained what you want to do ("highlighting of discriminant variables" isn't very informative). WHY do you want to use stepwise anything? Are you looking for the best set of explanatory variables to predict group membership? Are your data non-normal? Do your groups have heterogeneous covariance structures? Do you have vastly unequal mixture proportions? It sounds like you are interested in the function discrim(), which is part of S-Plus but not R. You've complained about the lack of documentation for qda() in Venables and Ripley (3rd edition?) and in the R documentation, but you haven't told us what you were looking for. Neither qda() nor lda() do "stepwise", and AFAIK, there is no stepwise logistic regression available in R. You might want multinomial logistic regression (multinom() in bundle VR), but that isn't stepwise either. In short, you're not getting much help because the questions you're asking aren't very well-formed. -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
I believe you can use stepAIC in the MASS library to do a stepwise logistic regression. As far as I know, the performance of lda typically depends critically on the multivariate normality of the data. Whether you get better results with lda or logistic regression depends on your particular situation. -roger _______________________________ UCLA Department of Statistics rpeng at stat.ucla.edu stat.ucla.edu/~rpeng On Thu, 23 May 2002, Daniel Amorèse wrote:> Hello, > Does logistic regression really provide better results than lda or qda ? > (my purpose is not classification but highlighting of discriminant > variables). > If this is true, where could I get an R script performing stepwise logistic > regression ? > Thanks > -- > Daniel AMORESE > Lab. M2C "Morphodynamique Continentale et Côtière" > UMR CNRS 6143 Caen/Rouen > Centre de Géomorphologie > UCBN (Université de Caen Basse-Normandie) > 14032 Caen Cedex > FRANCE > Ph: +33 (0)2 31 56 57 19 > Fax: +33 (0)2 31 56 57 57 > Courriel: amorese at geos.unicaen.fr > geos.unicaen.fr/perso/da/daperso.html > -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- > r-help mailing list -- Read ci.tuwien.ac.at/~hornik/R/R-FAQ.html > Send "info", "help", or "[un]subscribe" > (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch > _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._ >-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Daniel Amor?se wrote:> > Hello, > Does logistic regression really provide better results than lda or qda ? > (my purpose is not classification but highlighting of discriminant > variables). > If this is true, where could I get an R script performing stepwise logistic > regression ?stepAIC in package MASS (in bundle VR). Kjetil Halvorsen> Thanks > -- > Daniel AMORESE > Lab. M2C "Morphodynamique Continentale et C?ti?re" > UMR CNRS 6143 Caen/Rouen > Centre de G?omorphologie > UCBN (Universit? de Caen Basse-Normandie) > 14032 Caen Cedex > FRANCE > Ph: +33 (0)2 31 56 57 19 > Fax: +33 (0)2 31 56 57 57 > Courriel: amorese at geos.unicaen.fr > geos.unicaen.fr/perso/da/daperso.html > -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- > r-help mailing list -- Read ci.tuwien.ac.at/~hornik/R/R-FAQ.html > Send "info", "help", or "[un]subscribe" > (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch > _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Marc R. Feldesman
2002-May-24 21:02 UTC
[R] logistic regression or discriminant analysis ?
At 01:49 AM 5/24/2002, Daniel Amor?se took a glob of electronic fairy dust, mushed it together in odd and bizarre ways, and ruminated: >Le 2002.05.23 19:38, Marc Feldesman a ?crit : >> At 06:25 PM 5/23/2002 +0200, Daniel Amor?se wrote: >> >Hello, >What I have done: the correlation matrix tells me that many >variables are correlated. Thus, I performed a lda using only 5 >variables (this selection is arbitrary performed among uncorrelated >variables). The graphical output shows points clouds that are not >circular: this result may suggest difference in covariance >matrices, hence lda seems not to be the more suitable method for >separating groups. Elliptical point clouds do not, in themselves, suggest unequal covariance structures. If the major axes of the ellipses are all oriented in the same direction and are more-or-less parallel, you don't have much evidence of unequal covariance structure. Unless the covariance structures are really different, I've never seen much improvement using quadratic discriminant analysis. The problem with quadratic discriminant analysis, in my experience, is that it is very difficult to interpret the results in the context of the variables producing the group separations. I'd strongly recommend Professor Ripley's "Pattern Recognition and Neural Networks" (Cambridge University Press, 1996), as well as Hastie, Tibshirani, and Friedman's "The Elements of Statistical Learning" (2001, Cambridge University Press). Both will help you considerably with your problem. I also agree with Jon Baron that clustering techniques may be appropriate here. Dr. Marc R. Feldesman Professor and Chairman Anthropology Department - Portland State University email: feldesmanm at pdx.edu email: feldesman at attglobal.net fax: 503-725-3905 "Sometimes the lights are all shining on me, other times I can barely see, lately it's occurred to me, what a long strange trip it's been..." Jerry & the boys Powered by Tyrannochoerus - the 2.2 GHz WinXPP Box -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Le 2002.05.24 16:55, Marc R. Feldesman a ?crit :> At 01:49 AM 5/24/2002, Daniel Amor?se took a glob of electronic fairy > dust, > mushed it together in odd and bizarre ways, and ruminated: > >Le 2002.05.23 19:38, Marc Feldesman a ?crit : > >> At 06:25 PM 5/23/2002 +0200, Daniel Amor?se wrote: > >> >Hello, > >What I have done: the correlation matrix tells me that many > >variables are correlated. Thus, I performed a lda using only 5 > >variables (this selection is arbitrary performed among uncorrelated > >variables). The graphical output shows points clouds that are not > >circular: this result may suggest difference in covariance > >matrices, hence lda seems not to be the more suitable method for > >separating groups. > > Elliptical point clouds do not, in themselves, suggest unequal covariance > > structures. If the major axes of the ellipses are all oriented in the > same > direction and are more-or-less parallel, you don't have much evidence of > unequal covariance structure. Unless the covariance structures are > really > different, I've never seen much improvement using quadratic discriminant > analysis. The problem with quadratic discriminant analysis, in my > experience, is that it is very difficult to interpret the results in the > context of the variables producing the group separations.Ok, the elliptical point clouds I obtain are not oriented in the same direction, so I should give up with lda or qda> > I'd strongly recommend Professor Ripley's "Pattern Recognition and Neural > > Networks" (Cambridge University Press, 1996), as well as Hastie, > Tibshirani, and Friedman's "The Elements of Statistical Learning" (2001, > Cambridge University Press). Both will help you considerably with your > problem. >Thanks, for these references. What a pity the library of my university seems to be allergic to english-written books> I also agree with Jon Baron that clustering techniques may be appropriate > > here.Ok, my ideas about this kind of approach are confused (I do not need clustering because my groups are already defined). Some people told me about solving my problem using multinom() and stepAIC().....> > >Perhaps, qda should be used ? > >or logistic regression ? (this last method seems to be the more > >robust, independent to data properties). > >I know qda(), lda() or multinom() do not perform stepwise analysis, > >but, what I hope, is that some outputs from these functions can > >help in the selection of the most discriminatory variable subset. > >Thanks again for your help. > D. Amorese > > > Dr. Marc R. Feldesman > Professor and Chairman > Anthropology Department - Portland State University > email: feldesmanm at pdx.edu > email: feldesman at attglobal.net > fax: 503-725-3905 > > > "Sometimes the lights are all shining on me, other times I can barely > see, > lately it's occurred to me, what a long strange trip it's been..." Jerry > & > the boys > > > > Powered by Tyrannochoerus - the 2.2 GHz WinXPP Box > > >-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._