R help - Apr 2005 - negetative AIC values: How to compare models with negative AIC's

Dear,

When fitting the following model
knots <- 5
lrm.NDWI <- lrm(m.arson ~ rcs(NDWI,knots) 

I obtain the following result:

Logistic Regression Model

lrm(formula = m.arson ~ rcs(NDWI, knots))


Frequencies of Responses
  0   1 
666  35 

       Obs  Max Deriv Model L.R.       d.f.          P          C        Dxy
Gamma      Tau-a         R2      Brier 
       701      5e-07      34.49          4          0      0.777      0.553
0.563      0.053      0.147      0.045 

          Coef     S.E.    Wald Z P     
Intercept   -4.627   3.188 -1.45  0.1467
NDWI         5.333  20.724  0.26  0.7969
NDWI'        6.832  74.201  0.09  0.9266
NDWI''      10.469 183.915  0.06  0.9546
NDWI'''   -190.566 254.590 -0.75  0.4541

When analysing the glm fit of the same model

Call:  glm(formula = m.arson ~ rcs(NDWI, knots), x = T, y = T) 

Coefficients:
            (Intercept)     rcs(NDWI, knots)NDWI    rcs(NDWI, knots)NDWI'
rcs(NDWI, knots)NDWI''  rcs(NDWI, knots)NDWI'''  
                0.02067                  0.08441                 -0.54307
3.99550                -17.38573  

Degrees of Freedom: 700 Total (i.e. Null);  696 Residual
Null Deviance:      33.25 
Residual Deviance: 31.76        AIC: -167.7 

A negative AIC occurs!

How can the negative AIC from different models be compared with each other?
Is this result logical? Is the lowest AIC still correct?


Thanks,
Jan

_______________________________________________________________________
ir. Jan Verbesselt 
Research Associate 
Lab of Geomatics Engineering K.U. Leuven
Vital Decosterstraat 102. B-3000 Leuven Belgium 
Tel: +32-16-329750   Fax: +32-16-329760
http://gloveg.kuleuven.ac.be/

AICs (like log-likelihoods) can be positive or negative.
However, you fitted a Gaussian and not a binomial glm (as lrm does if 
m.arson is binary).

For a discrete response with the usual dominating measure (counting 
measure) the log-likelihood is negative and hence the AIC is positive,
but not in general (and it is matter of convention even there).

In any case, Akaike only suggested comparing AIC for nested models, no one
suggests comparing continuous and discrete models.

On Fri, 15 Apr 2005, Jan Verbesselt wrote:
>
> Dear,
>
> When fitting the following model
> knots <- 5
> lrm.NDWI <- lrm(m.arson ~ rcs(NDWI,knots)
>
> I obtain the following result:
>
> Logistic Regression Model
>
> lrm(formula = m.arson ~ rcs(NDWI, knots))
>
>
> Frequencies of Responses
>  0   1
> 666  35
>
>       Obs  Max Deriv Model L.R.       d.f.          P          C        Dxy
> Gamma      Tau-a         R2      Brier
>       701      5e-07      34.49          4          0      0.777      0.553
> 0.563      0.053      0.147      0.045
>
>          Coef     S.E.    Wald Z P
> Intercept   -4.627   3.188 -1.45  0.1467
> NDWI         5.333  20.724  0.26  0.7969
> NDWI'        6.832  74.201  0.09  0.9266
> NDWI''      10.469 183.915  0.06  0.9546
> NDWI'''   -190.566 254.590 -0.75  0.4541
>
> When analysing the glm fit of the same model
>
> Call:  glm(formula = m.arson ~ rcs(NDWI, knots), x = T, y = T)
>
> Coefficients:
>            (Intercept)     rcs(NDWI, knots)NDWI    rcs(NDWI,
knots)NDWI'
> rcs(NDWI, knots)NDWI''  rcs(NDWI, knots)NDWI'''
>                0.02067                  0.08441                 -0.54307
> 3.99550                -17.38573
>
> Degrees of Freedom: 700 Total (i.e. Null);  696 Residual
> Null Deviance:      33.25
> Residual Deviance: 31.76        AIC: -167.7
>
> A negative AIC occurs!
>
> How can the negative AIC from different models be compared with each other?
> Is this result logical? Is the lowest AIC still correct?
-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

Jan Verbesselt wrote:
> Dear,
> 
> When fitting the following model
> knots <- 5
> lrm.NDWI <- lrm(m.arson ~ rcs(NDWI,knots) 
> 
> I obtain the following result:
> 
> Logistic Regression Model
> 
> lrm(formula = m.arson ~ rcs(NDWI, knots))
> 
> 
> Frequencies of Responses
>   0   1 
> 666  35 
> 
>        Obs  Max Deriv Model L.R.       d.f.          P          C       
Dxy
> Gamma      Tau-a         R2      Brier 
>        701      5e-07      34.49          4          0      0.777     
0.553
> 0.563      0.053      0.147      0.045 
> 
>           Coef     S.E.    Wald Z P     
> Intercept   -4.627   3.188 -1.45  0.1467
> NDWI         5.333  20.724  0.26  0.7969
> NDWI'        6.832  74.201  0.09  0.9266
> NDWI''      10.469 183.915  0.06  0.9546
> NDWI'''   -190.566 254.590 -0.75  0.4541
> 
> When analysing the glm fit of the same model
> 
> Call:  glm(formula = m.arson ~ rcs(NDWI, knots), x = T, y = T) 
> 
> Coefficients:
>             (Intercept)     rcs(NDWI, knots)NDWI    rcs(NDWI,
knots)NDWI'
> rcs(NDWI, knots)NDWI''  rcs(NDWI, knots)NDWI'''  
>                 0.02067                  0.08441                 -0.54307
> 3.99550                -17.38573  
> 
> Degrees of Freedom: 700 Total (i.e. Null);  696 Residual
> Null Deviance:      33.25 
> Residual Deviance: 31.76        AIC: -167.7 
> 
> A negative AIC occurs!
> 
> How can the negative AIC from different models be compared with each other?
> Is this result logical? Is the lowest AIC still correct?
I'm not sure about this particular example but in general there is no 
problem with a negative AIC or a negative deviance just as there is no 
problem with a positive log-likelihood.  It is a common misconception 
that the log-likelihood must be negative.  If the likelihood is derived 
from a probability density it can quite reasonably exceed 1 which means 
that log-likelihood is positive, hence the deviance and the AIC are 
negative.

If you believe that comparing AICs is a good way to choose a model then 
it would still be the case that the (algebraically) lower AIC is preferred.