R help - Mar 2009 - Test Set Deviances

Dear all

I'm building logistic regression models using glm() and then
validating their predictive ability on a test set using predict() to
get the probabilities. I understand how to attain the deviance of the
model fitted to the training set. But is there a way to get the
deviance within the test set?

Cheers

Tom

-- 
Thomas Allen
Department of Biochemistry
University of Otago
710 Cumberland street
Box 56
Dunedin
New Zealand
Work: (+64)3 479 5123
Mobile: 027 321 4269
email1: hedbag at gmail.com
email2: allth025 at student.otago.ac.nz

Only the insane have strength enough to prosper and only those who
prosper can truly judge what is sane

Thomas Allen wrote:
> Dear all
> 
> I'm building logistic regression models using glm() and then
> validating their predictive ability on a test set using predict() to
> get the probabilities. I understand how to attain the deviance of the
> model fitted to the training set. But is there a way to get the
> deviance within the test set?
> 
> Cheers
> 
> Tom
> 
This is essentially the index "D" reported by validate.lrm in the
Design
package.  Don't use just one test set unless it has more than 15,000 
subjects in it.  validate.lrm will do resampling for you.

Frank
-- 
Frank E Harrell Jr   Professor and Chair           School of Medicine
                      Department of Biostatistics   Vanderbilt University