Hi, I am using logistic regression model named lrm(Design) Rite now I was using Area Under Curve (AUC) for testing my model. But, now I have to calculate precision/recall of the model on test cases. For lrm, precision and recal would be simply defined with the help of 2 terms below: True Positive (TP) - Number of test cases where class 1 is given probability>= 0.5.False Negative (FP) - Number of test cases where class 0 is given probability >= 0.5. Precision = TP / (TP + FP) Recall = TP / ( Number of Positive Samples in test data) Any help is appreciated. I an write a long code with for loops and all, but is there any inbuild function or just few commands that would do the task. regards, Nitin [[alternative HTML version deleted]]
nitin jindal wrote:> Hi, > > I am using logistic regression model named lrm(Design) > > Rite now I was using Area Under Curve (AUC) for testing my model. But, now I > have to calculate precision/recall of the model on test cases. > For lrm, precision and recal would be simply defined with the help of 2 > terms below: > True Positive (TP) - Number of test cases where class 1 is given probability >> = 0.5. > False Negative (FP) - Number of test cases where class 0 is given > probability >= 0.5.Why 0.5?> > Precision = TP / (TP + FP) > Recall = TP / ( Number of Positive Samples in test data)Those are improper scoring rules that can be tricked. If the outcome is rare (say 0.02 incidence) you could just predict that no one will have the outcome and be correct 0.98 of the time. I suggest validating the model for discrimination (e.g., AUC) and calibration. Frank> > Any help is appreciated. > > I an write a long code with for loops and all, but is there any inbuild > function or just few commands that would do the task. > > regards, > Nitin > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >-- Frank E Harrell Jr Professor and Chair School of Medicine Department of Biostatistics Vanderbilt University
Hi. Thnx a lot. I will try that. nitin On 1/24/07, Tobias Sing <tobias.sing@mpi-sb.mpg.de> wrote:> > Maybe ROCR might help you. > You can visualize the prec/rec-trade-off across the range of all cutoffs: > assuming your numerical predictions are in scores and the true class > labels are in classes: > pred <- prediction( scores, classes ) > perf <- performance(pred, 'rec','prec') > plot(perf) > > HTH, > Tobias > > On 1/24/07, nitin jindal <nitin.jindal@gmail.com> wrote: > > Hi, > > > > I am using logistic regression model named lrm(Design) > > > > Rite now I was using Area Under Curve (AUC) for testing my model. But, > now I > > have to calculate precision/recall of the model on test cases. > > For lrm, precision and recal would be simply defined with the help of 2 > > terms below: > > True Positive (TP) - Number of test cases where class 1 is given > probability > > >= 0.5. > > False Negative (FP) - Number of test cases where class 0 is given > > probability >= 0.5. > > > > Precision = TP / (TP + FP) > > Recall = TP / ( Number of Positive Samples in test data) > > > > Any help is appreciated. > > > > I an write a long code with for loops and all, but is there any inbuild > > function or just few commands that would do the task. > > > > regards, > > Nitin > > > > [[alternative HTML version deleted]] > > > > ______________________________________________ > > R-help@stat.math.ethz.ch mailing list > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > > and provide commented, minimal, self-contained, reproducible code. > > > > > -- > Tobias Sing > Computational Biology and Applied Algorithmics > Max Planck Institute for Informatics > Saarbrucken, Germany > Phone: +49 681 9325 315 > Fax: +49 681 9325 399 > http://www.tobiassing.net >[[alternative HTML version deleted]]