Hello - I am using glmnet to generate a model for multiple cohorts i. For each i, I run 5 separate models, each with a different x variable. I want to compare the fit statistic for each i and x combination. When I use auc, the output is in some cases is < .5 (.49). In addition, if I compare mean MSE (with upper and lower bounds) ... there is no difference across my various x variables, but mean AUC (with upper and lower bounds) shows differentiation. My basic questions are, should I not expect AUC to lie between .5 and 1 and, which model fit measurement is most appropriate for comparing across models (if the various statistics are producing a somewhat inconsistent story). Thanks in advance for any advice. Below is my code and sample output for AUC/MSE. xc <- split(dataS$P1_retained, dataS$TotalHours_R) yc <- split(dataS$x, dataS$TotalHours_R) for (i in 1:length(yc)) { fit=cv.glmnet(as.matrix(yc[[i]]), y=xc[[i]], alpha=.05, type="mse", nfolds=10, standardize=TRUE,family="binomial") c_output c(i,fit$cvlo[fit$lambda==fit$lambda.1se],fit$cvm[fit$lambda==fit$lambda.1se], fit$cvup[fit$lambda==fit$lambda.1se]) names(c_output) = names(output_x) output_x = rbind(output_x, t(c_output)) fit1=cv.glmnet(as.matrix(yc[[i]]), y=xc[[i]], alpha=.05, type="auc", nfolds=10, standardize=TRUE,family="binomial") c_output1 c(i,fit1$cvlo[fit1$lambda==fit1$lambda.1se],fit1$cvm[fit1$lambda==fit1$lambda.1se], fit1$cvup[fit1$lambda==fit1$lambda.1se]) names(c_output1) = names(output_x1) output_x1 = rbind(output_x1, t(c_output1)) fit2=cv.glmnet(as.matrix(yc[[i]]), y=xc[[i]], alpha=.05, type="class", nfolds=10, standardize=TRUE,family="binomial") c_output2 c(i,fit2$cvlo[fit2$lambda==fit2$lambda.1se],fit2$cvm[fit2$lambda==fit2$lambda.1se], fit2$cvup[fit2$lambda==fit2$lambda.1se]) names(c_output2) = names(output_x2) output_x2 = rbind(output_x2, t(c_output2)) } COHORT LB_MSE_X MEAN_MSE_X UB_MSE_X LB_AUC_X MEAN_AUC_X UB_AUC_X LB_CLASS_X MEAN_CLASS_X UB_CLASS_X 0 0.44 0.44 0.44 0.50 0.50 0.50 0.33 0.33 0.33 1 0.42 0.42 0.42 0.51 0.51 0.52 0.30 0.30 0.30 2 0.40 0.40 0.40 0.50 0.50 0.50 0.28 0.28 0.28 3 0.36 0.37 0.37 0.51 0.51 0.51 0.24 0.24 0.24 4 0.35 0.35 0.35 0.51 0.51 0.51 0.22 0.23 0.23 5 0.33 0.33 0.33 0.51 0.51 0.52 0.21 0.21 0.21 6 0.32 0.32 0.32 0.51 0.51 0.51 0.20 0.20 0.20 7 0.30 0.31 0.31 0.52 0.52 0.52 0.19 0.19 0.19 8 0.29 0.29 0.30 0.52 0.52 0.52 0.18 0.18 0.18 9 0.28 0.29 0.29 0.52 0.52 0.52 0.17 0.17 0.17 10 0.28 0.28 0.28 0.52 0.53 0.53 0.17 0.17 0.17 [[alternative HTML version deleted]]