search for: deviance

A maybe trivial and stupid question: In the case of a lm or glm fit, it is quite informative (to me) to have a look to the null deviance and the residual deviance of a model. This is generally provided in the print method or the summary, eg: Null Deviance: 658.8 Residual Deviance: 507.3 and (a bit simpled minded) I like to think that the proportion of deviance 'explained' by the model is (658.8-507.3)/658.8 = 23% I...

Dear R users, I read your methods of extracting the variance explained by each predictor in different places. My question is: using the method you suggested, the sum of the deviance explained by all terms is not equal to the deviance explained by the full model. Could you tell me what caused such problem? > set.seed(0) > n<-400 > x1 <- runif(n, 0, 1) > ## to see problem with not fixing smoothing parameters > ## remove the `##' from the next line,...

Dear all, I am using gamlss (Package gamlss version 4.0-0, R version 2.10.1, Windows XP Service Pack 3 on a HP EliteBook) to relate bird counts to habit variables. However, most models fail because “the global deviance is increasing” and I am not sure what causes this behaviour. The dataset consists of counts of birds (duck) and 5 habit variables measured in the field (n= 182). The dependent variable (the number of ducks counted)’suffers’ from zero-inflation and overdisperion: > proportion_non_zero <- (sum...

...ión de los insectos (picadas) Factores: 1) Especie de insecto (especie), 2) Estado de desarrollo de los insectos (edad) y 3) Estado de desarrollo de frutos (estado). Modelo: MLG con distribución binomial negativa como componente aleatorio. Estrategia de selección de modelo: Evaluación del cambio de deviance y su valor P al ir agregando términos Cuando corro el modelo Nulo *Nulo<-glm.nb(picadas~1, data=nochoice)* obtengo: *Null deviance: 263.50 on 299 degrees of freedom* *Residual deviance: 263.50 on 299 degrees of freedom* *AIC: 794.67* Pero al correr las otras versiones posibles, obtengo u...

Dear R-users, To obtain the percentage of deviance explained when fitting a gam model using the mgcv library is straightforward: summary(object.gam) $dev.expl or alternatively, using the deviance (deviance(object.gam)) of the null and the fitted models, and then using 1 minus the quotient of deviances. However, when a gamm (generalizad...

...the significance of each term (F, p-value) and to the "wiggliness" of the fitted smooth (edf), but (as far as I understand) there is no information on the proportion of variance explained. One alternative may be to fit alternative models without each term, and calculate the reduction in deviance. For example: m1=gam(y~s(x1) + s(x2)) # Full model m2=gam(y~s(x2)) m3=gam(y~s(x1)) ddev1=deviance(m1)-deviance(m2) ddev2=deviance(m1)-deviance(m3) Here, ddev1 would measure the relative proportion of the variability in y explained by x1, and ddev2 would do the same for x2. Does this sound lik...

Hi: I'm working with multinomial models with library nnet, and I'm trying to get the explained deviance (pseudo R^2) of my models. I am assuming that: pseudo R^2= 1 - dev(model) / dev (null) where dev(model) is the deviance for the fitted model and dev(null) is the deviance for the null model (with the intercept only). library(nnet) full.model<- multinom(cbind(factor1, factor2 ,., factor5)...

...g Crawley's book closely and am wondering if there is an accepted standard for how much is too much overdispersion? (e.g. change in AIC has an accepted standard of 2). In the example, he fits several models, binomial and quasibinomial and then accepts the quasibinomial. The output for residual deviance and df does not change between these two models, however, he accepts the quasibinomial. Is there a different calculation that I missed that actually provides an overdispersion factor (as in SAS?) My code and output are below, given the example in the book, these data are WAY overdispersed .....do...

...ding the mother (fam) of the fawns as random factor. I want to use AIC to select the best model (if no other model selection criterias are suggested). the syntax looks like this: > mod <- lmer(sfox ~ roefoxratio + (1|fam), data=manu2, family=binomial) The output looks ok, except that the deviance is extremely high (1.798e+308). > mod Generalized linear mixed model fit using PQL Formula: sfox ~ roefoxratio + (1 | fam) Data: manu2 Family: binomial(logit link) AIC BIC logLik deviance 1.797693e+308 1.797693e+308 -8.988466e+307 1.797693e+308 Rando...

Running a binary logit model on the data df <- data.frame(y=sample(letters[1:3], 100, repl=T), x=rnorm(100)) reveals some residual deviance: summary(glm(y ~ ., data=df, family=binomial("logit"))) However, running a multinomial model on that data (multinom, nnet) reveals a residual deviance: summary(multinom(y ~ ., data=df)) On page 203, the MASS book says that "here the deviance is comparing with the model that cor...

...at more complex problem than given in Simon Wood's example below. My full model has more than two smooths as well as factor variables as in this simplified example: b <- gam(y~fv1+s(x1)+s(x2)+s(x3)) Judging from Simon's example, my guess is to fit reduced models to get components of deviance as follows: b1 <- gam(y~s(x1)+s(x2)+s(x3),sp=b$sp) b2 <- gam(y~fv1+s(x2)+s(x3),sp=b$sp[-1]) b3 <- gam(y~fv1+s(x1)+s(x3),sp=b$sp[-2]) b4 <- gam(y~fv1+s(x1)+s(x2),sp=b$sp[-3]) b0 <- gam(y~1) And then proceed as in Simon's example, i.e. (deviance(b1)-deviance(b))/deviance(b0) #...

Hello, The question looks like simple. It's probably even stupid. But I spent several hours searching Internet, downloaded tons of papers, where deviance is mentioned and... And haven't found an answer. Well, it is clear for me the using of entropy when I split some node of a classification tree. The sense is clear, because entropy is an old good measure of how uniform is distribution. And we want, for sure, the distribution to be uniform, rep...

Greetings all, I'm a newcomer to Bayesian stats, and I'm trying to calculate the Deviance Information Criterion "by hand" from some MCMC output. However, having consulted several sources, I am left confused as to the exact terms to use. The most common formula can be written as DIC = 2*Mean(Deviance over the whole sampled posterior distribution) - Deviance(Mean posterio...

...------------- > summary(d) y x1 x2 x3 0:54 0:50 0:64 0:78 1:46 1:50 1:36 1:22 > fit <- glm(y ~ x1 + x2 + x3, data=d, family=binomial(link=logit)) > summary(fit) Call: glm(formula = y ~ x1 + x2 + x3, family = binomial(link = logit), data = d) Deviance Residuals: Min 1Q Median 3Q Max -1.6503 -1.0220 -0.7284 0.9965 1.7069 Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) -0.3772 0.3721 -1.014 0.3107 x11 -0.8144 0.4422 -1.842 0.0655 . x21 0.9226...

Dear all, I am new to R and my question may be trivial to you... I am doing a GLM with binomial errors to compare proportions of species in different categories of seed sizes (4 categories) between 2 sites. In the model summary the residual deviance is much higher than the degree of freedom (Residual deviance: 153.74 on 4 degrees of freedom) and even after correcting for overdispersion by using a quasibinomial error structure instead of binomial the residual deviance does not change. Is this a data problem and I cannot use this statistic or...

Hi all Consider the family function, as used by glm. The help page says the value of the family object is a list, one element of which is the following: dev.resids function giving the deviance residuals as a function of (y, mu, wt). But reading any of the family functions (eg poisson) shows that dev.resids is a function that computes the *square* of the deviance residuals (at least, by every definition I know of). Further, the deviance residuals returned by residuals.glm are ess...

Hi all, I constructed a GAM model with a linear term and two smooth terms, all of them statistically significant but the intercept was not significant. The adjusted r2 of this model is 0.572 and the deviance 65.3. I decided to run the model again without intercept, so I used in R the following instruction: regression= gam(dependent~ +linear_independent +s(smooth_independent_1) +s(smooth_independent_2) -1,data=D) All the independent variables were significant. The adjusted r2 of this model is 0.552 and...

...############## set.seed(666) dat <- data.frame(x = rep((-10):10, each = 5), w = rep(1:5, 21)) dat <- transform(dat, y = rbinom(x, size = w, prob = pcauchy(1 + 2*x))) modFit <- glm(y/w ~ x, quasi(link = cauchit, variance = "mu(1-mu)"), dat, weights = w, trace = T) Deviance = 309.2785 Iterations - 1 Deviance = 3257.501 Iterations - 2 Deviance = 1043.455 Iterations - 3 .. Deviance = 1733.824 Iterations - 24 Deviance = 1665.487 Iterations - 25 Warning message: algorithm did not converge in: glm.fit(x = X, ... ################################################# A com...

...y*sqrt(W), Xs<- sqrt(W)*cbind(1,x) So we can see weights here for lm means the scale for X and y. But for glm and gls I try > glm1 Call: glm(formula = y ~ x) Coefficients: (Intercept) x 0.1183 7.3075 Degrees of Freedom: 1242 Total (i.e. Null); 1241 Residual Null Deviance: 1049000 Residual Deviance: 28210 AIC: 7414 > glm2 Call: glm(formula = y ~ x, weights = W) Coefficients: (Intercept) x 0.1955 7.3053 Degrees of Freedom: 1242 Total (i.e. Null); 1241 Residual Null Deviance: 1548000 Residual Deviance: 44800 AIC: 1...

I am comparing the Splus and R fits of a simple glm. In the following, foo is generated from rbinom with size = 20 p = 0.5. The coefficients (and SE's0 of the fitted models are the same, but the estimated deviances are quite different. Could someone please tell me why they are so different? I am using R version 1.3.1 and Splus 2000 release 3 on windows 2000. ++++++++++++++++++++++ foo <- c(9, 4, 10, 7, 11, 13, 8, 6, 11, 14, 11, 10, 7, 9, 13, 7, 9, 6, 10, 10) foo.glm <- glm(cbind(f...