R help - Apr 2006 - lme: null deviance, deviance due to the random effects, residual deviance

A maybe trivial and stupid question:

In the case of a lm or glm fit, it is quite informative (to me) to have 
a look to the null deviance and the residual deviance of a model. This 
is generally provided in the print method or the summary, eg:

Null Deviance:      658.8
Residual Deviance: 507.3

and (a bit simpled minded) I like to think that the proportion of 
deviance 'explained' by the model is (658.8-507.3)/658.8 = 23%

In the case of lme models, is it possible and reasonable to try and get the:
- null deviance
- the total deviance due to the the random effect(s)
- the residual deviance?

With the idea that Null deviance = Fixed effects + Random Effects + 
Residuals

If yes how to do it ? A lme object provides the following:

 > names(glm6)
 [1] "modelStruct"  "dims"         "contrasts"   
"coefficients"
 [5] "varFix"       "sigma"        "apVar"       
"logLik"
 [9] "numIter"      "groups"       "call"        
"method"
[13] "fitted"       "residuals"    "fixDF"       
"family"

so no $null.deviance and $deviance elements as in glm objects...

I tried to find out an answer on R-help & Pineihro & Bates (2000). 
Partial success only:

- null deviance: Response: possibly yes: see 
http://tolstoy.newcastle.edu.au/R/help/05/12/17796.html (Spencer 
Graves). The (null?) deviance is -2*logLik(mylme), but a personnal trial 
with some glm objects did not led to the same numbers that the one given 
by the print.glm method...

- the deviance due to the the random effect(s). I was supposing that the 
coefficients given by ranef(mylme) may be an entry... but beyond this, I 
guess those coefficients must be weighed in some way... which is a far 
beyond my capacities in this matter...

- residual deviance. I was supposing that it may be 
sum(residuals(mylme)^2). With some doubts as far as I feel that I am 
thinking sum of squares estimation in the context of likelihood and 
deviance estimations... So  most likely irrelevant. Moreover, in the 
case I was exploring, this quantity is much larger than the null 
deviance computed as above...

Any hint appreciated,

Patrick Giraudoux

As far as I know, the term "deviance" has no standard definition.  A 
good, fairly common definition (I think) is that the deviance is "up to 
[an additive] constant, minus twice the maximised log-likelihood. Where 
sensible, the constant is chosen so that a saturated model has deviance 
zero." 
("http://finzi.psych.upenn.edu/R/library/gnm/html/gnm.html".)
Because of this "constant", the "the proportion of deviance
'explained'
by the model" in not a well defined concept.  I found this definition 
using RSiteSearch("deviance define").  However, even this definition
is
not used consistently;  it's not even used for 'deviance.lm', which
I
discovered using methods("deviance") and methods("logLik")
followed by
'getAnywhere("deviance.lm"), etc.

	  This is not a "trivial and stupid question".  Instead, it's
connected
to subtle issues in statistical methods, and this reply may contribute 
more obfuscation than enlightenment.  If you describe some more specific 
application where you might want to use something like this and what you 
are trying to achieve, you might get a more useful reply.

	  hope this helps,
	  spencer graves

Patrick Giraudoux wrote:
> A maybe trivial and stupid question:
> 
> In the case of a lm or glm fit, it is quite informative (to me) to have 
> a look to the null deviance and the residual deviance of a model. This 
> is generally provided in the print method or the summary, eg:
> 
> Null Deviance:      658.8
> Residual Deviance: 507.3
> 
> and (a bit simpled minded) I like to think that the proportion of 
> deviance 'explained' by the model is (658.8-507.3)/658.8 = 23%
> 
> In the case of lme models, is it possible and reasonable to try and get
the:
> - null deviance
> - the total deviance due to the the random effect(s)
> - the residual deviance?
> 
> With the idea that Null deviance = Fixed effects + Random Effects + 
> Residuals
> 
> If yes how to do it ? A lme object provides the following:
> 
>  > names(glm6)
>  [1] "modelStruct"  "dims"        
"contrasts"    "coefficients"
>  [5] "varFix"       "sigma"        "apVar"   
"logLik"
>  [9] "numIter"      "groups"       "call"    
"method"
> [13] "fitted"       "residuals"    "fixDF"   
"family"
> 
> so no $null.deviance and $deviance elements as in glm objects...
> 
> I tried to find out an answer on R-help & Pineihro & Bates (2000). 
> Partial success only:
> 
> - null deviance: Response: possibly yes: see 
> http://tolstoy.newcastle.edu.au/R/help/05/12/17796.html (Spencer 
> Graves). The (null?) deviance is -2*logLik(mylme), but a personnal trial 
> with some glm objects did not led to the same numbers that the one given 
> by the print.glm method...
> 
> - the deviance due to the the random effect(s). I was supposing that the 
> coefficients given by ranef(mylme) may be an entry... but beyond this, I 
> guess those coefficients must be weighed in some way... which is a far 
> beyond my capacities in this matter...
> 
> - residual deviance. I was supposing that it may be 
> sum(residuals(mylme)^2). With some doubts as far as I feel that I am 
> thinking sum of squares estimation in the context of likelihood and 
> deviance estimations... So  most likely irrelevant. Moreover, in the 
> case I was exploring, this quantity is much larger than the null 
> deviance computed as above...
> 
> Any hint appreciated,
> 
> Patrick Giraudoux
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide!
http://www.R-project.org/posting-guide.html