R help - Jun 2009 - rpart - the xval argument in rpart.control and in xpred.rpart

Dear R users,
I'm working with the rpart package and want to evaluate the performance of 
user defined split  functions.
I have some problems in understanding the meaning of the xval argument in 
the two functions rpart.control and xpred.rpart. In the former it is defined 
as the number of cross-validations while in the latter it is defined as the 
number of cross-validation groups. If I am correct this means that for 
xpred.rpart, xval denots the number V of one V-fold cross validation while 
for rpart.control it denotes the number of times that the cross-validation 
is performed. Am I right ? If so, how many subsets compose the learning 
sample when performing one of the xval cross-validations ? Is it a 10-fold 
cross-validation repeated xval=10 times ?
Thank you for your help

Paolo

Paolo Radaelli
Dipartimento di Metodi Quantitativi per le Scienze Economiche ed Aziendali
Facolt? di Economia
Universit? degli Studi di Milano-Bicocca
Via Bicocca degli Arcimboldi, 8
20126 Milano
Italy
e-mail paolo.radaelli at unimib.it
Tel +39 02 6448 3163
Fax +39 02 6448 3105

> I have some problems in understanding the meaning of the xval argument in 
> the two functions rpart.control and xpred.rpart. In the former it is
defined
> as the number of cross-validations while in the latter it is defined as the
> number of cross-validation groups. 
  It is the same thing.  If xval=10 then the data is divided into 10 disjoint 
groups.  A model is fit with group 1 left out and that model is used to predict 
the observations in group 1; then a model is fit with group 2 left out; then 
group 3, ...
   So 10 groups = 10 fits of the model.
   
   
   
   	Terry Therneau

Usually 10-fold cross validation is performed more than once to get an 
estimate of the misclassification rate thus I thought "number of 
cross-validations" was different from the "number of cross-validation 
groups". So, if I want to perform 10-fold cross-validation more than once 
(say 5) in order to estimate the miscalssification rate I have to run 
xpred.rpart 5 times ?
Thanks
Paolo

>> I have some problems in understanding the meaning of the xval argument
in
>> the two functions rpart.control and xpred.rpart. In the former it is 
>> defined
>> as the number of cross-validations while in the latter it is defined as
>> the
>> number of cross-validation groups.
>  It is the same thing.  If xval=10 then the data is divided into 10 
> disjoint
> groups.  A model is fit with group 1 left out and that model is used to 
> predict
> the observations in group 1; then a model is fit with group 2 left out; 
> then
> group 3, ...
>   So 10 groups = 10 fits of the model.


Actually I thought that in rpart.control
>
>
>
>   Terry Therneau
>
>

I have this *exact* same confusion. 

Adding to this is the fact that Everitt and Hothorn in their book, HSAUR,
say that setting xval=100 gives "100 runs of 10-fold cross-validation"
(1st
ed., page 136).

Is this actually 1 run of 100-fold cross-validation? 

For large xval, doing multiple cross-validations is not super important. But
I would want to perform multiple cross-validataion with different partitions
of the data when xval is moderate or small wrt the size of the data set. In
that case do we need to do as Paolo suggests?




Paolo Radaelli wrote:
> 
> Usually 10-fold cross validation is performed more than once to get an 
> estimate of the misclassification rate thus I thought "number of 
> cross-validations" was different from the "number of
cross-validation
> groups". So, if I want to perform 10-fold cross-validation more than
once
> (say 5) in order to estimate the miscalssification rate I have to run 
> xpred.rpart 5 times ?
> Thanks
> Paolo
> 
> 
>>> I have some problems in understanding the meaning of the xval
argument
>>> in
>>> the two functions rpart.control and xpred.rpart. In the former it
is
>>> defined
>>> as the number of cross-validations while in the latter it is
defined as
>>> the
>>> number of cross-validation groups.
>>  It is the same thing.  If xval=10 then the data is divided into 10 
>> disjoint
>> groups.  A model is fit with group 1 left out and that model is used to
>> predict
>> the observations in group 1; then a model is fit with group 2 left out;
>> then
>> group 3, ...
>>   So 10 groups = 10 fits of the model.
> 
> 
> 
> Actually I thought that in rpart.control
> 
>>
>>
>>
>>   Terry Therneau
>>
>>
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
> 
> 
-- 
View this message in context:
http://www.nabble.com/Re%3A-rpart---the-xval-argument-in-rpart.control-and-in-xpred.rpart-tp23942907p25408496.html
Sent from the R help mailing list archive at Nabble.com.