Dear R users! Does someone know about any algorithms / packages in R, that perform classification / regression / decision trees using multivariate splits? I have done some research, but I found nothing. Packages "tree" and "rpart" seem only to be able to do CART with univariate splits. Thank you for your help! B?lint -- Cz?cz B?lint PhD hallgat? BCE KTK Talajtan ?s V?zgazd?lkod?s Tansz?k 1118 Budapest, Vill?nyi ?t 29-43.
On Mon, 19 Jun 2006, B?lint Cz?cz wrote:> Dear R users! > > Does someone know about any algorithms / packages in R, that perform > classification / regression / decision trees using multivariate > splits? > > I have done some research, but I found nothing. Packages "tree" and > "rpart" seem only to be able to do CART with univariate splits. >have a look at the machine learning task view on CRAN, which will point you to package `RWeka'. HTH, Torsten> Thank you for your help! > > B?lint > > -- > Cz?cz B?lint > PhD hallgat? > BCE KTK Talajtan ?s V?zgazd?lkod?s Tansz?k > 1118 Budapest, Vill?nyi ?t 29-43. > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html > >
If by "multivariate split" the OP meant splitting on combinations of covariates (instead of multi-way split on a single covariate at a time), there aren't that many methods published (AFAIK). All the ones I know about are in the cheminformatics area: RP-SA and generalizations/extensions of it. No R package I know of can do it. Best, Andy From: Torsten Hothorn> > On Mon, 19 Jun 2006, B?lint Cz?cz wrote: > > > Dear R users! > > > > Does someone know about any algorithms / packages in R, > that perform > > classification / regression / decision trees using multivariate > > splits? > > > > I have done some research, but I found nothing. Packages "tree" and > > "rpart" seem only to be able to do CART with univariate splits. > > > > have a look at the machine learning task view on CRAN, which > will point you to package `RWeka'. > > HTH, > > Torsten > > > Thank you for your help! > > > > B?lint > > > > -- > > Cz?cz B?lint > > PhD hallgat? > > BCE KTK Talajtan ?s V?zgazd?lkod?s Tansz?k > > 1118 Budapest, Vill?nyi ?t 29-43. > > > > ______________________________________________ > > R-help at stat.math.ethz.ch mailing list > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide! > > http://www.R-project.org/posting-guide.html > > > > >
Glenn De'ath published a paper in 'Ecology' several years ago and included S-Plus functions in the archives. I haven't looked at the files, so I'm not sure what modifications would be necessary for R. De'ath, G. 2002. Multivariate regression trees: a new technique for modeling species--environment relationships. Ecology 83:1105-1117. Archives: http://www.esapubs.org/archive/ecol/E083/017/ Scott> Date: Mon, 19 Jun 2006 16:17:40 +0200 > From: " B?lint Cz?cz " <elatine@gmail.com> > Subject: [R] multivariate splits > To: r-help <r-help@stat.math.ethz.ch> > Message-ID: <fab4bcf70606190717r45a2030dg@mail.gmail.com> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > Dear R users! > > Does someone know about any algorithms / packages in R, that perform > classification / regression / decision trees using multivariate > splits? > > I have done some research, but I found nothing. Packages "tree" and > "rpart" seem only to be able to do CART with univariate splits. > > Thank you for your help! > > B?lint > > -- > Cz?cz B?lint > PhD hallgat? > BCE KTK Talajtan ?s V?zgazd?lkod?s Tansz?k > 1118 Budapest, Vill?nyi ?t 29-43.--------------------------------- [[alternative HTML version deleted]]
From: Gavin Simpson> > On Tue, 2006-06-20 at 08:12 -0700, Scott Rollins wrote: > > Glenn De'ath published a paper in 'Ecology' several years ago and > > included S-Plus functions in the archives. I haven't looked at the > > files, so I'm not sure what modifications would be necessary for R. > > > > De'ath, G. 2002. Multivariate regression trees: a new technique for > > modeling species--environment relationships. Ecology 83:1105-1117. > > > > Archives: http://www.esapubs.org/archive/ecol/E083/017/ > > > > Scott > > Glenn's code is in package mvpart on CRAN. > > Not sure if the OP wanted this or not, but the multivariate > nature of mvpart is in allowing multivariate responses, > common in ecological data analysis. The fitting is still done > using rpart (actually a modified version of rpart to allow > for the multivariate response). > > GavinThe OP's reply to my follow-up to Torsten's message seems to indicate that he has univariate response. He wants something that can split on linear combinations of predictors, as described in the CART book, I believe. What I thought he wanted was something that finds some optimal subset of predictors to split each node. I am not aware of any open source implementations of tree algorithms that does linear combination splits, perhaps others know better. I suppose Torsten's double bagging (in the ipred package) sort of does that, but in an ensemble rather than a single tree. Andy> > > > > > > Date: Mon, 19 Jun 2006 16:17:40 +0200 > > > From: " B?lint Cz?cz " <elatine at gmail.com> > > > Subject: [R] multivariate splits > > > To: r-help <r-help at stat.math.ethz.ch> > > > Message-ID: <fab4bcf70606190717r45a2030dg at mail.gmail.com> > > > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > > > > > Dear R users! > > > > > > Does someone know about any algorithms / packages in R, > that perform > > > classification / regression / decision trees using multivariate > > > splits? > > > > > > I have done some research, but I found nothing. Packages > "tree" and > > > "rpart" seem only to be able to do CART with univariate splits. > > > > > > Thank you for your help! > > > > > > B?lint > > > > > > -- > > > Cz?cz B?lint > > > PhD hallgat? > > > BCE KTK Talajtan ?s V?zgazd?lkod?s Tansz?k > > > 1118 Budapest, Vill?nyi ?t 29-43. > > > > > > --------------------------------- > > > > > > [[alternative HTML version deleted]] > > > > ______________________________________________ > > R-help at stat.math.ethz.ch mailing list > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide! > > http://www.R-project.org/posting-guide.html > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! > http://www.R-project.org/posting-guide.html > >
Thanks to everyone for the replies to my question. There are several good algorithms to make decision trees with multivariate splits (sometimes called oblique trees), some are even freeware (QUEST, CRUISE...), but most are not open-sourced. Unfortunately there seems to be no good choice for oblique trees in R now. Mvpart seems to be really fancy, although it cannot handle splits on the linear combination of the covariates. Rweka might be useful, but it still takes some time for me to get used to its "language". I'll see where I get. Thanks again and bye: B?lint -- Cz?cz B?lint PhD hallgat? BCE KTK Talajtan ?s V?zgazd?lkod?s Tansz?k 1118 Budapest, Vill?nyi ?t 29-43.