Lars Kunert

2009-Mar-16 22:24 UTC

### [R] the effect of blocking on the size of confidence intervals - analysis using lme and lmer

This is a follow-up mail of "the effect of blocking on the size of confidence intervals - analysis using aov". In both mails I pursue the idea of using blocking factors in order to reduce the width of confidence intervals. My dataset comprises, a quantitative response variable, namely: "response", and three categorical eplanatory variables, namely: "method", "target", and "query". The method variable has 4 levels, namely: XXX.pg.trained, XXX.rpg.trained, MACCS, and FTREES. The target variable has 28 levels. The query variable has 84 levels. For every target three queries are selected. Thus "query" is nested in "target". The design is completely balanced. "response" "method" "target" "query" 0.68 "XXX.pg.trained" "06243" "06243:218379" 0.96 "XXX.pg.trained" "06243" "06243:218384" 0.72 "XXX.pg.trained" "06243" "06243:246192" 0.7696 "XXX.pg.trained" "06245" "06245:235027" ... For the complete dataset see the attached file data.txt Using lm and lme, I analyze three models:> m0.lm = lm ( response ~ method -1, data = d) > m1.lme = lme( response ~ method -1, random = ~ 1 | target, data = d) > m2.lme = lme( response ~ method -1, random = ~ 1 | query , data = d)Using lmer the calls are:> m1.lmer = lmer( response ~ method -1 + ( 1 | target ), data = d)...> summary.lm( m0.lm )... Estimate Std. Error t value Pr(>|t|) methodFTREES 0.34163 0.03221 10.61 <2e-16 *** methodMACCS 0.45525 0.03221 14.13 <2e-16 *** methodXXX.pg.trained 0.49498 0.03221 15.37 <2e-16 *** methodXXX.rpg.trained 0.45049 0.03221 13.99 <2e-16 *** ... Residual standard error: 0.2952 on 332 degrees of freedom ...> summary( m1.lme )... Value Std.Error DF t-value p-value methodFTREES 0.3416262 0.04871665 305 7.012514 0 methodMACCS 0.4552452 0.04871665 305 9.344756 0 methodXXX.pg.trained 0.4949833 0.04871665 305 10.160455 0 methodXXX.rpg.trained 0.4504917 0.04871665 305 9.247180 0 ...> summary( m2.lme )... Value Std.Error DF t-value p-value methodFTREES 0.3416262 0.03220994 249 10.60624 0 methodMACCS 0.4552452 0.03220994 249 14.13369 0 methodXXX.pg.trained 0.4949833 0.03220994 249 15.36741 0 methodXXX.rpg.trained 0.4504917 0.03220994 249 13.98611 0 ... Thus the confidence interval for "methodFTREES" is given by: 0.34163 +- qt( 0.025, 332 )*0.03221 = ( 0.2782686, 0.4049914 ) # m0.lm 0.3416262 +- qt( 0.025, 305 )*0.04871665 = ( 0.2457629, 0.4374895 ) # m1.lme 0.3416262 +- qt( 0.025, 249 )*0.03220994 = ( 0.2781875, 0.4050649 ) # m2.lme Thus the interval for m1.lme is larger than the interval for m0.lm which contradicts the utility of blocking... Using lmer instead of lme, identical results can be derived. Note that the intervals for lmer can alternatively be derived by:> intervals( m1.lme )... methodFTREES 0.2457629 0.3416262 0.4374895 methodMACCS 0.3593820 0.4552452 0.5511085 methodXXX.pg.trained 0.3991201 0.4949833 0.5908466 methodXXX.rpg.trained 0.3546284 0.4504917 0.5463549 ... I have the following questions: - How are the std. errors of lme calculated? It seems like the std errors of lme/lmer are more conservative than the std errors reported by lm. This would explain for the increase of the std error (and hence the size of the confidence interval) from m0.lm to m1.lme. - Is its possible to use lme without a random term, or alternatively with a trivial random term? - Is the overall approach: use of blocking factors to reduce the residual varaince, and hence to decrease the width of the confidence intervals valid? Using lmer, alternatively to the calculation of CIs, credible intervals can be calculated:> m1.sample = mcmcsamp( m1.lmer, n =10000 ) > m1.ci = apply( m1.sample, 2, function(x)quantile(x, prob=c(.025,0.975))) > m1.ci[,"methodFTREES"]2.5% 97.5% 0.2405115 0.4424920 ...> m2.ci[,"methodFTREES"]2.5% 97.5% 0.2767626 0.4042516 For my intended use: the comparison of the average performance of different methods credible intervals a probably as good as CI. However, how do I calculate credible intervals for m0.lm??? Thanks for your help, Lars -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: data.txt URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20090316/0697521e/attachment-0003.txt>