R help - Oct 2008 - Credit Scoring Model - SPEC (specificity) and SENS (sensitivity)

Dear R helpers,

Hi I am working on credit scoring model using logistic regression. I have main
sample of 42500 clentes and based on their status as regards to defaulted / non
- defaulted, I have genereted the probability of default.

I have a hold out sample of 5000 clients. I have calculated (1) No of correctly
classified goods Gg, (2) No of correcly classified Bads Bg and also (3) number
of wrongly classified bads (Gb) and (4) number of wrongly classified goods (Bg).

My prolem is how to interpret these results? What I have arrived at are the
absolute figures. Using these I hav ecalculated Specificity (SPEC) and
sensitivity (SENS) as

SPEC = Bb / (Bb + Gg)

and SENS = Gg / (Gg + Bg)


With regards

Maithili

Maithili Shiva wrote:
> Dear R helpers,
> 
> Hi I am working on credit scoring model using logistic regression. I have
main sample of 42500 clentes and based on their status as regards to defaulted /
non - defaulted, I have genereted the probability of default.
> 
> I have a hold out sample of 5000 clients. I have calculated (1) No of
correctly classified goods Gg, (2) No of correcly classified Bads Bg and also
(3) number of wrongly classified bads (Gb) and (4) number of wrongly classified
goods (Bg).
> 
> My prolem is how to interpret these results? What I have arrived at are the
absolute figures. Using these I hav ecalculated Specificity (SPEC) and
sensitivity (SENS) as
> 
> SPEC = Bb / (Bb + Gg)
> 
> and SENS = Gg / (Gg + Bg)
> 
> 
> With regards
> 
> Maithili
Sensitivity and specificity have no usefulness in your situation as they 
are in reverse time order (condition on the unknown and fail to 
condition on what was already known).  Your test sample is too small. 
You are not addressing absolute calibration through precise 
high-resolution methods.  My book Regression Modeling Strategies goes 
into this.

Frank
> 
-- 
Frank E Harrell Jr   Professor and Chair           School of Medicine
                      Department of Biostatistics   Vanderbilt University