Using AUC for discrete predictor variables with inly two levels
doesn't seem very sensible. What are you planning to to with this
measure?
--
David.
On Jun 1, 2011, at 8:47 PM, <Jin.Li at ga.gov.au> <Jin.Li at
ga.gov.au> wrote:
> Hi all,
> I used the following code and data to get auc values for two sets of
> predictions:
> library(caret)
>> table(predicted1, trainy)
> trainy
> hard soft
> 1 27 0
> 2 11 99
>> aucRoc(roc(predicted1, trainy))
> [1] 0.5
>
>
>> table(predicted2, trainy)
> trainy
> hard soft
> 1 27 2
> 2 11 97
>> aucRoc(roc(predicted2, trainy))
> [1] 0.8451621
>
> predicted1:
> 1 1 2 2 2 1 2 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 1 2 2 2 2 2 2 2 2 2 1 2
> 2 2 2 2 1 2 2 2 2 1 1 2 2 2 2 2 1 2 2 2 2 2 1 2 2 2 2 2 2 2 2 2 2 2
> 2 2 2 1 2 2 2 2 2 2 2 1 2 2 2 2 2 1 1 1 2 2 1 1 1 2 2 2 2 2 1 1 2 2
> 2 2 2 2 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
>
> predicted2:
> 1 1 2 1 2 1 2 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 1 2 2 2 2 2 2 2 2 1 1 2
> 2 2 2 2 1 2 2 2 2 1 1 2 2 2 2 2 1 2 2 2 2 2 1 2 2 2 2 2 2 2 2 2 2 2
> 2 2 2 1 2 2 2 2 2 2 2 1 2 2 2 2 2 1 1 1 2 2 1 1 1 2 2 2 2 2 1 1 2 2
> 2 2 2 2 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
>
> trainy:
> hard hard hard soft soft hard hard hard hard soft soft soft soft
> soft soft hard soft soft soft soft soft soft hard soft soft soft
> soft soft soft soft soft soft hard soft soft soft soft soft hard
> soft soft soft soft hard hard soft soft soft hard soft hard soft
> soft soft soft soft hard soft soft soft soft soft soft soft soft
> hard soft soft soft soft soft hard soft soft soft soft soft soft
> soft hard soft soft soft hard hard hard hard hard soft soft hard
> hard hard soft hard soft soft soft hard hard soft soft soft soft
> soft hard hard hard hard hard hard hard soft soft soft soft soft
> soft soft soft soft soft soft soft soft soft soft soft hard soft
> soft soft soft soft soft soft soft
> Levels: hard soft
>
>> Sys.info()
> sysname
> release version nodename
> "Windows" "XP"
"build
> 2600, Service Pack 3" "PC-60772"
> machine
> "x86"
>
> I would expect predicted1 is more accurate that the predicted2. But
> the auc values show an opposite. I was wondering whether this is a
> bug or I have done something wrong. Thanks for your help in advance!
>
> Cheers,
>
> Jin
> ____________________________________
> Jin Li, PhD
> Spatial Modeller/Computational Statistician
> Marine & Coastal Environment
> Geoscience Australia
> GPO Box 378, Canberra, ACT 2601, Australia
>
> Ph: 61 (02) 6249 9899; email:
> jin.li at ga.gov.au<mailto:jin.li at ga.gov.au>
> _______________________________________
>
>
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
David Winsemius, MD
West Hartford, CT