I'm sorry I haven't boiled this down to a more-minimal example yet,
but ...
I'm working on an S3 method (tidy.merMod, in the 'broom' package).
It
normally handles 'merMod' objects from the lme4 package, but I'm
trying
to make it handle 'merModLmerTest' objects from the lmerTest package
too.
The merModLmerTest class inherits (as an S4) class from the merMod
class. The important difference (for my current purposes) is that
summary.merMod returns an object containing a coefficient table
*without* degrees or freedom or p-values, while the summary method for
merModLmerTest objects returns one *with* both of those columns.
Because merModLmerTest inherits from merMod, calling tidy(obj) on a
merModLmerTest object does get you into broom:::tidy.merMod. Within
that function, though, calling summary() appears to call summary.merMod
instead of merModLmerTest, so I don't get the additional columns I want.
Ideas/directions for further diagnoses? Is this an S3/S4 confusion,
or something funny about the way the broom and merModLmerTest package
are talking to each other, or ... ?
cheers
Ben Bolker
----
library(lme4)
fm1 <- lmer(Reaction~Days+(Days|Subject),sleepstudy)
library(lmerTest)
fm1B <- as(fm1,"merModLmerTest")
coef(summary(fm1B)) ## table with df and p values
coef(lme4:::summary.merMod(fm1B)) ## table without df and p values
is(fm1B,"merMod") ## TRUE
library(broom)
tidy(fm1B)
## hard to show what's going on without debugging through the function:
## the issue occurs here:
## https://github.com/dgrtwo/broom/blob/master/R/lme4_tidiers.R#L94
I can't replicate this behaviour with a trivial method ...
silly <- function(x, ...) {
UseMethod("silly")
}
silly.merMod <- function(object, ...) {
coef(summary(object))
}
silly(fm1B)
environment(silly) <- environment(tidy)
silly(fm1B)
>>>>> Ben Bolker <bbolker at gmail.com> >>>>> on Fri, 15 Jul 2016 16:45:50 -0400 writes:> I'm sorry I haven't boiled this down to a more-minimal example yet, > but ... > I'm working on an S3 method (tidy.merMod, in the 'broom' package). It > normally handles 'merMod' objects from the lme4 package, but I'm trying > to make it handle 'merModLmerTest' objects from the lmerTest package too. > The merModLmerTest class inherits (as an S4) class from the merMod > class. The important difference (for my current purposes) is that > summary.merMod returns an object containing a coefficient table > *without* degrees or freedom or p-values, while the summary method for > merModLmerTest objects returns one *with* both of those columns. > Because merModLmerTest inherits from merMod, calling tidy(obj) on a > merModLmerTest object does get you into broom:::tidy.merMod. Within > that function, though, calling summary() appears to call summary.merMod > instead of merModLmerTest, so I don't get the additional columns I want. > Ideas/directions for further diagnoses? Is this an S3/S4 confusion, > or something funny about the way the broom and merModLmerTest package > are talking to each other, or ... ? I'm not sure, if I see your problem when I execute the reproducible code below ... > ---- > library(lme4) > fm1 <- lmer(Reaction~Days+(Days|Subject),sleepstudy) > library(lmerTest) > fm1B <- as(fm1,"merModLmerTest") > coef(summary(fm1B)) ## table with df and p values > coef(lme4:::summary.merMod(fm1B)) ## table without df and p values > is(fm1B,"merMod") ## TRUE > library(broom) > tidy(fm1B) Part of what I see from above, is > coef(summary(fm1B)) ## table with df and p values Estimate Std. Error df t value Pr(>|t|) (Intercept) 251.40510 6.824556 17.00002 36.838310 0.000000e+00 Days 10.46729 1.545789 17.00000 6.771485 3.263788e-06 > tidy(fm1B) term estimate std.error statistic group 1 (Intercept) 251.40510485 6.824556 36.838310 fixed 2 Days 10.46728596 1.545789 6.771485 fixed 3 sd_(Intercept).Subject 24.74044768 NA NA Subject 4 sd_Days.Subject 5.92213326 NA NA Subject 5 cor_(Intercept).Days.Subject 0.06555134 NA NA Subject 6 sd_Observation.Residual 25.59181589 NA NA Residual > which seems consistent at least: You get std.error and statistic for those two terms that you get 'df' and 'p-value from "the good method" of summary(.) ? Of course, I'm using the CRAN versions of the packages, not github ones, and I see that we end up using half of tidyVerse ... which does not necessarily keep debugging easy.. Martin :> sessionInfo()R version 3.3.1 Patched (2016-07-16 r70928) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Fedora 22 (Twenty Two) locale: [1] LC_CTYPE=de_CH.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 [4] LC_COLLATE=de_CH.UTF-8 LC_MONETARY=en_US.UTF-8 LC_MESSAGES=de_CH.UTF-8 [7] LC_PAPER=de_CH.UTF-8 LC_NAME=C LC_ADDRESS=C [10] LC_TELEPHONE=C LC_MEASUREMENT=de_CH.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] graphics grDevices datasets stats utils methods base other attached packages: [1] broom_0.4.1 lmerTest_2.0-32 lme4_1.1-12 Matrix_1.2-3 fortunes_1.5-3 [6] sfsmisc_1.1-0 loaded via a namespace (and not attached): [1] Rcpp_0.12.5 compiler_3.3.1 nloptr_1.0.4 RColorBrewer_1.1-2 [5] plyr_1.8.4 tools_3.3.1 rpart_4.1-10 tibble_1.1 [9] gtable_0.2.0 nlme_3.1-128 lattice_0.20-33 psych_1.6.6 [13] DBI_0.4-1 parallel_3.3.1 gridExtra_2.2.1 stringr_1.0.0 [17] dplyr_0.5.0 cluster_2.0.3 grid_3.3.1 nnet_7.3-12 [21] data.table_1.9.6 R6_2.1.2 survival_2.39-5 foreign_0.8-66 [25] latticeExtra_0.6-28 minqa_1.2.4 Formula_1.2-1 tidyr_0.5.1 [29] reshape2_1.4.1 ggplot2_2.1.0 magrittr_1.5 Hmisc_3.17-4 [33] scales_0.4.0 MASS_7.3-45 splines_3.3.1 assertthat_0.1 [37] mnormt_1.5-4 colorspace_1.2-6 stringi_1.1.1 acepack_1.3-3.3 [41] munsell_0.4.3 chron_2.3-47>> ## hard to show what's going on without debugging through the function: > ## the issue occurs here: > ## https://github.com/dgrtwo/broom/blob/master/R/lme4_tidiers.R#L94 > I can't replicate this behaviour with a trivial method ... > silly <- function(x, ...) { > UseMethod("silly") > } > silly.merMod <- function(object, ...) { > coef(summary(object)) > } > silly(fm1B) > environment(silly) <- environment(tidy) > silly(fm1B) > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel
I'm not doing a good job at explaining. See inline below ... On 16-07-18 11:02 AM, Martin Maechler wrote:>>>>>> Ben Bolker <bbolker at gmail.com> >>>>>> on Fri, 15 Jul 2016 16:45:50 -0400 writes:akram > > > I'm sorry I haven't boiled this down to a more-minimal example yet, > > but ... > > > I'm working on an S3 method (tidy.merMod, in the 'broom' package). It > > normally handles 'merMod' objects from the lme4 package, but I'm trying > > to make it handle 'merModLmerTest' objects from the lmerTest package too. > > > The merModLmerTest class inherits (as an S4) class from the merMod > > class. The important difference (for my current purposes) is that > > summary.merMod returns an object containing a coefficient table > > *without* degrees or freedom or p-values, while the summary method for > > merModLmerTest objects returns one *with* both of those columns. > > > Because merModLmerTest inherits from merMod, calling tidy(obj) on a > > merModLmerTest object does get you into broom:::tidy.merMod. Within > > that function, though, calling summary() appears to call summary.merMod > > instead of merModLmerTest, so I don't get the additional columns I want. > > > Ideas/directions for further diagnoses? Is this an S3/S4 confusion, > > or something funny about the way the broom and merModLmerTest package > > are talking to each other, or ... ? > > I'm not sure, if I see your problem when I execute the > reproducible code below ... > > > > ---- > > library(lme4) > > fm1 <- lmer(Reaction~Days+(Days|Subject),sleepstudy) > > library(lmerTest) > > fm1B <- as(fm1,"merModLmerTest") > > coef(summary(fm1B)) ## table with df and p values > > coef(lme4:::summary.merMod(fm1B)) ## table without df and p values > > > is(fm1B,"merMod") ## TRUE > > > library(broom) > > tidy(fm1B) > > Part of what I see from above, is > > > coef(summary(fm1B)) ## table with df and p values > Estimate Std. Error df t value Pr(>|t|) > (Intercept) 251.40510 6.824556 17.00002 36.838310 0.000000e+00 > Days 10.46729 1.545789 17.00000 6.771485 3.263788e-06 > > > tidy(fm1B) > term estimate std.error statistic group > 1 (Intercept) 251.40510485 6.824556 36.838310 fixed > 2 Days 10.46728596 1.545789 6.771485 fixed > 3 sd_(Intercept).Subject 24.74044768 NA NA Subject > 4 sd_Days.Subject 5.92213326 NA NA Subject > 5 cor_(Intercept).Days.Subject 0.06555134 NA NA Subject > 6 sd_Observation.Residual 25.59181589 NA NA Residual > > > > which seems consistent at least: You get std.error and statistic > for those two terms that you get 'df' and 'p-value from "the > good method" of summary(.)This is as expected. What's going on here is that I'm trying to get the p-value into the tidied object. You really have to debug *into* the function to see the problem. I've tried to remedy this a bit in my fork by adding a 'debug' argument: devtools::install_github("bbolker/broom") ## ... re-load package or re-run session ... tidy(fm1B,debug=TRUE) produces output from coef(summary(x)): Estimate Std. Error t value (Intercept) 251.40510 6.824556 36.838310 Days 10.46729 1.545789 6.771485 that is, when coef(summary(x)) is run *inside* the tidy.merMod function, it calls summary.merMod rather than the summary method for merModLmerTest objects ...> > ? > > Of course, I'm using the CRAN versions of the packages, not > github ones, and I see that we end up using half of tidyVerse > ... which does not necessarily keep debugging easy.. >Sorry about that. broom is fairly heavily integrated with the tidyverse ...> Martin : > >> sessionInfo() > R version 3.3.1 Patched (2016-07-16 r70928) > Platform: x86_64-pc-linux-gnu (64-bit) > Running under: Fedora 22 (Twenty Two) > > locale: > [1] LC_CTYPE=de_CH.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 > [4] LC_COLLATE=de_CH.UTF-8 LC_MONETARY=en_US.UTF-8 LC_MESSAGES=de_CH.UTF-8 > [7] LC_PAPER=de_CH.UTF-8 LC_NAME=C LC_ADDRESS=C > [10] LC_TELEPHONE=C LC_MEASUREMENT=de_CH.UTF-8 LC_IDENTIFICATION=C > > attached base packages: > [1] graphics grDevices datasets stats utils methods base > > other attached packages: > [1] broom_0.4.1 lmerTest_2.0-32 lme4_1.1-12 Matrix_1.2-3 fortunes_1.5-3 > [6] sfsmisc_1.1-0 > > loaded via a namespace (and not attached): > [1] Rcpp_0.12.5 compiler_3.3.1 nloptr_1.0.4 RColorBrewer_1.1-2 > [5] plyr_1.8.4 tools_3.3.1 rpart_4.1-10 tibble_1.1 > [9] gtable_0.2.0 nlme_3.1-128 lattice_0.20-33 psych_1.6.6 > [13] DBI_0.4-1 parallel_3.3.1 gridExtra_2.2.1 stringr_1.0.0 > [17] dplyr_0.5.0 cluster_2.0.3 grid_3.3.1 nnet_7.3-12 > [21] data.table_1.9.6 R6_2.1.2 survival_2.39-5 foreign_0.8-66 > [25] latticeExtra_0.6-28 minqa_1.2.4 Formula_1.2-1 tidyr_0.5.1 > [29] reshape2_1.4.1 ggplot2_2.1.0 magrittr_1.5 Hmisc_3.17-4 > [33] scales_0.4.0 MASS_7.3-45 splines_3.3.1 assertthat_0.1 > [37] mnormt_1.5-4 colorspace_1.2-6 stringi_1.1.1 acepack_1.3-3.3 > [41] munsell_0.4.3 chron_2.3-47 >> > > > > ## hard to show what's going on without debugging through the function: > > ## the issue occurs here: > > ## https://github.com/dgrtwo/broom/blob/master/R/lme4_tidiers.R#L94 > > > I can't replicate this behaviour with a trivial method ... > > > silly <- function(x, ...) { > > UseMethod("silly") > > } > > silly.merMod <- function(object, ...) { > > coef(summary(object)) > > } > > silly(fm1B) > > environment(silly) <- environment(tidy) > > silly(fm1B) > > > ______________________________________________ > > R-devel at r-project.org mailing list > > https://stat.ethz.ch/mailman/listinfo/r-devel >