On Wednesday 20 April 2005 00:17, array chip wrote:> Hi all,
>
> In Tibshirani's PNAS paper about nearest shrunken
> centroid analysis of microarrays (PNAS vol 99:6567),
> they used cross validation to choose the amount of
> shrinkage used in the model, and then test the
> performance of the model with the cross-validated
> shrinkage in separate independent testing set. If I
> don't have the luxury of having independent testing
> set, can I just use the cross validation performance
> as the performance estimate? In other words, can I use
> the same single cross-validation to both choose the
> value of the parameter (amount of shrinkage in this
> case) and estimate the performance which was based on
> the value of the parameter chosen by the same
> cross-validation? I kind of feel awkward by getting
> both on a single cross validation, because it seems
> like I used the dataset in training set manner. Am I
> wrong/right?
That error rate is probably optimistic, because as you
say> cross-validation? I kind of feel awkward by getting
> both on a single cross validation, because it seems
> like I used the dataset in training set manner. Am I
However, you can easily wrap the whole pam procedure within an outer-loop of
cross validation or bootstrap. (This problem is not that different from, say,
using knn and selecting k using cross-validation; or selecting the number of
genes to use with cross-validation, etc. You should then assess the error
rate of your procedure).
R.>
> Thanks!
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide!
> http://www.R-project.org/posting-guide.html
--
Ram??n D??az-Uriarte
Bioinformatics Unit
Centro Nacional de Investigaciones Oncol??gicas (CNIO)
(Spanish National Cancer Center)
Melchor Fern??ndez Almagro, 3
28029 Madrid (Spain)
Fax: +-34-91-224-6972
Phone: +-34-91-224-6900
http://ligarto.org/rdiaz
PGP KeyID: 0xE89B3462
(http://ligarto.org/rdiaz/0xE89B3462.asc)
**NOTA DE CONFIDENCIALIDAD** Este correo electr??nico, y en su caso los ficheros
adjuntos, pueden contener informaci??n protegida para el uso exclusivo de su
destinatario. Se proh??be la distribuci??n, reproducci??n o cualquier otro tipo
de transmisi??n por parte de otra persona que no sea el destinatario. Si usted
recibe por error este correo, se ruega comunicarlo al remitente y borrar el
mensaje recibido.
**CONFIDENTIALITY NOTICE** This email communication and any attachments may
contain confidential and privileged information for the sole use of the
designated recipient named above. Distribution, reproduction or any other use of
this transmission by any party other than the intended recipient is prohibited.
If you are not the intended recipient please contact the sender and delete all
copies.