thr3ads.net - R devel - [Rd] summary() dispatch puzzle [Jul 2016]

If this information is useful, please help other people find it:
Share via:

Ben Bolker

2016-Jul-15 20:45 UTC

[Rd] summary() dispatch puzzle

I'm sorry I haven't boiled this down to a more-minimal example yet,
but ...

  I'm working on an S3 method (tidy.merMod, in the 'broom' package).
It
normally handles 'merMod' objects from the lme4 package, but I'm
trying
to make it handle 'merModLmerTest' objects from the lmerTest package
too.

  The merModLmerTest class inherits (as an S4) class from the merMod
class.  The important difference (for my current purposes) is that
summary.merMod returns an object containing a coefficient table
*without* degrees or freedom or p-values, while the summary method for
merModLmerTest objects returns one *with* both of those columns.

 Because merModLmerTest inherits from merMod, calling tidy(obj) on a
merModLmerTest object does get you into broom:::tidy.merMod.  Within
that function, though, calling summary() appears to call summary.merMod
instead of merModLmerTest, so I don't get the additional columns I want.

  Ideas/directions for further diagnoses?  Is this an S3/S4 confusion,
or something funny about the way the broom and merModLmerTest package
are talking to each other, or ... ?

  cheers
    Ben Bolker


----
library(lme4)
fm1 <- lmer(Reaction~Days+(Days|Subject),sleepstudy)
library(lmerTest)
fm1B <- as(fm1,"merModLmerTest")
coef(summary(fm1B))               ## table with df and p values
coef(lme4:::summary.merMod(fm1B)) ## table without df and p values

is(fm1B,"merMod")  ## TRUE

library(broom)
tidy(fm1B)
## hard to show what's going on without debugging through the function:
## the issue occurs here:
## https://github.com/dgrtwo/broom/blob/master/R/lme4_tidiers.R#L94

I can't replicate this behaviour with a trivial method ...

silly <- function(x, ...) {
    UseMethod("silly")
}
silly.merMod <- function(object, ...) {
    coef(summary(object))
}
silly(fm1B)
environment(silly) <- environment(tidy)
silly(fm1B)

Martin Maechler

2016-Jul-18 15:02 UTC

head link

[Rd] summary() dispatch puzzle

>>>>> Ben Bolker <bbolker at gmail.com>
>>>>>     on Fri, 15 Jul 2016 16:45:50 -0400 writes:
    > I'm sorry I haven't boiled this down to a more-minimal example
yet,
    > but ...

    > I'm working on an S3 method (tidy.merMod, in the 'broom'
package). It
    > normally handles 'merMod' objects from the lme4 package, but
I'm trying
    > to make it handle 'merModLmerTest' objects from the lmerTest
package too.

    > The merModLmerTest class inherits (as an S4) class from the merMod
    > class.  The important difference (for my current purposes) is that
    > summary.merMod returns an object containing a coefficient table
    > *without* degrees or freedom or p-values, while the summary method for
    > merModLmerTest objects returns one *with* both of those columns.

    > Because merModLmerTest inherits from merMod, calling tidy(obj) on a
    > merModLmerTest object does get you into broom:::tidy.merMod.  Within
    > that function, though, calling summary() appears to call summary.merMod
    > instead of merModLmerTest, so I don't get the additional columns I
want.

    > Ideas/directions for further diagnoses?  Is this an S3/S4 confusion,
    > or something funny about the way the broom and merModLmerTest package
    > are talking to each other, or ... ?

I'm not sure, if I see your problem when I execute the
reproducible code below ... 


    > ----
    > library(lme4)
    > fm1 <- lmer(Reaction~Days+(Days|Subject),sleepstudy)
    > library(lmerTest)
    > fm1B <- as(fm1,"merModLmerTest")
    > coef(summary(fm1B))               ## table with df and p values
    > coef(lme4:::summary.merMod(fm1B)) ## table without df and p values

    > is(fm1B,"merMod")  ## TRUE

    > library(broom)
    > tidy(fm1B)

Part of what I see from above, is

    > coef(summary(fm1B))               ## table with df and p values
		 Estimate Std. Error       df   t value     Pr(>|t|)
    (Intercept) 251.40510   6.824556 17.00002 36.838310 0.000000e+00
    Days         10.46729   1.545789 17.00000  6.771485 3.263788e-06

    > tidy(fm1B)
			      term     estimate std.error statistic    group
    1                  (Intercept) 251.40510485  6.824556 36.838310    fixed
    2                         Days  10.46728596  1.545789  6.771485    fixed
    3       sd_(Intercept).Subject  24.74044768        NA        NA  Subject
    4              sd_Days.Subject   5.92213326        NA        NA  Subject
    5 cor_(Intercept).Days.Subject   0.06555134        NA        NA  Subject
    6      sd_Observation.Residual  25.59181589        NA        NA Residual
    > 

which seems consistent at least: You get std.error and statistic
for those two terms that you get 'df' and 'p-value from "the
good method" of summary(.)

?

Of course, I'm using the CRAN versions of the packages, not
github ones, and I see that we end up using half of tidyVerse
... which does not necessarily keep debugging easy..

Martin :
> sessionInfo()R version 3.3.1 Patched (2016-07-16 r70928)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Fedora 22 (Twenty Two)

locale:
 [1] LC_CTYPE=de_CH.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8
 [4] LC_COLLATE=de_CH.UTF-8     LC_MONETARY=en_US.UTF-8   
LC_MESSAGES=de_CH.UTF-8
 [7] LC_PAPER=de_CH.UTF-8       LC_NAME=C                  LC_ADDRESS=C
[10] LC_TELEPHONE=C             LC_MEASUREMENT=de_CH.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] graphics  grDevices datasets  stats     utils     methods   base     

other attached packages:
[1] broom_0.4.1     lmerTest_2.0-32 lme4_1.1-12     Matrix_1.2-3   
fortunes_1.5-3
[6] sfsmisc_1.1-0  

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.5         compiler_3.3.1      nloptr_1.0.4       
RColorBrewer_1.1-2
 [5] plyr_1.8.4          tools_3.3.1         rpart_4.1-10        tibble_1.1
 [9] gtable_0.2.0        nlme_3.1-128        lattice_0.20-33     psych_1.6.6
[13] DBI_0.4-1           parallel_3.3.1      gridExtra_2.2.1     stringr_1.0.0
[17] dplyr_0.5.0         cluster_2.0.3       grid_3.3.1          nnet_7.3-12
[21] data.table_1.9.6    R6_2.1.2            survival_2.39-5     foreign_0.8-66
[25] latticeExtra_0.6-28 minqa_1.2.4         Formula_1.2-1       tidyr_0.5.1
[29] reshape2_1.4.1      ggplot2_2.1.0       magrittr_1.5        Hmisc_3.17-4
[33] scales_0.4.0        MASS_7.3-45         splines_3.3.1       assertthat_0.1
[37] mnormt_1.5-4        colorspace_1.2-6    stringi_1.1.1       acepack_1.3-3.3
[41] munsell_0.4.3       chron_2.3-47       > 

    > ## hard to show what's going on without debugging through the
function:
    > ## the issue occurs here:
    > ## https://github.com/dgrtwo/broom/blob/master/R/lme4_tidiers.R#L94

    > I can't replicate this behaviour with a trivial method ...

    > silly <- function(x, ...) {
    > UseMethod("silly")
    > }
    > silly.merMod <- function(object, ...) {
    > coef(summary(object))
    > }
    > silly(fm1B)
    > environment(silly) <- environment(tidy)
    > silly(fm1B)

    > ______________________________________________
    > R-devel at r-project.org mailing list
    > https://stat.ethz.ch/mailman/listinfo/r-devel

Ben Bolker

2016-Jul-18 16:08 UTC

head link

[Rd] summary() dispatch puzzle

I'm not doing a good job at explaining.  See inline below ...

On 16-07-18 11:02 AM, Martin Maechler wrote:>>>>>> Ben Bolker <bbolker at gmail.com>
>>>>>>     on Fri, 15 Jul 2016 16:45:50 -0400 writes:akram
> 
>     > I'm sorry I haven't boiled this down to a more-minimal
example yet,
>     > but ...
> 
>     > I'm working on an S3 method (tidy.merMod, in the
'broom' package). It
>     > normally handles 'merMod' objects from the lme4 package,
but I'm trying
>     > to make it handle 'merModLmerTest' objects from the
lmerTest package too.
> 
>     > The merModLmerTest class inherits (as an S4) class from the merMod
>     > class.  The important difference (for my current purposes) is that
>     > summary.merMod returns an object containing a coefficient table
>     > *without* degrees or freedom or p-values, while the summary method
for
>     > merModLmerTest objects returns one *with* both of those columns.
> 
>     > Because merModLmerTest inherits from merMod, calling tidy(obj) on
a
>     > merModLmerTest object does get you into broom:::tidy.merMod. 
Within
>     > that function, though, calling summary() appears to call
summary.merMod
>     > instead of merModLmerTest, so I don't get the additional
columns I want.
> 
>     > Ideas/directions for further diagnoses?  Is this an S3/S4
confusion,
>     > or something funny about the way the broom and merModLmerTest
package
>     > are talking to each other, or ... ?
> 
> I'm not sure, if I see your problem when I execute the
> reproducible code below ... 
> 
> 
>     > ----
>     > library(lme4)
>     > fm1 <- lmer(Reaction~Days+(Days|Subject),sleepstudy)
>     > library(lmerTest)
>     > fm1B <- as(fm1,"merModLmerTest")
>     > coef(summary(fm1B))               ## table with df and p values
>     > coef(lme4:::summary.merMod(fm1B)) ## table without df and p values
> 
>     > is(fm1B,"merMod")  ## TRUE
> 
>     > library(broom)
>     > tidy(fm1B)
> 
> Part of what I see from above, is
> 
>     > coef(summary(fm1B))               ## table with df and p values
> 		 Estimate Std. Error       df   t value     Pr(>|t|)
>     (Intercept) 251.40510   6.824556 17.00002 36.838310 0.000000e+00
>     Days         10.46729   1.545789 17.00000  6.771485 3.263788e-06
> 
>     > tidy(fm1B)
> 			      term     estimate std.error statistic    group
>     1                  (Intercept) 251.40510485  6.824556 36.838310   
fixed
>     2                         Days  10.46728596  1.545789  6.771485   
fixed
>     3       sd_(Intercept).Subject  24.74044768        NA        NA 
Subject
>     4              sd_Days.Subject   5.92213326        NA        NA 
Subject
>     5 cor_(Intercept).Days.Subject   0.06555134        NA        NA 
Subject
>     6      sd_Observation.Residual  25.59181589        NA        NA
Residual
>     > 
> 
> which seems consistent at least: You get std.error and statistic
> for those two terms that you get 'df' and 'p-value from
"the
> good method" of summary(.)
  This is as expected.  What's going on here is that I'm trying to get
the p-value into the tidied object.   You really have to debug *into*
the function to see the problem.  I've tried to remedy this a bit in my
fork by adding a 'debug' argument:

devtools::install_github("bbolker/broom")
## ... re-load package or re-run session ...
tidy(fm1B,debug=TRUE)

produces

output from coef(summary(x)):
             Estimate Std. Error   t value
(Intercept) 251.40510   6.824556 36.838310
Days         10.46729   1.545789  6.771485


that is, when coef(summary(x)) is run *inside* the tidy.merMod function,
it calls summary.merMod rather than the summary method for
merModLmerTest objects ...
> 
> ?
> 
> Of course, I'm using the CRAN versions of the packages, not
> github ones, and I see that we end up using half of tidyVerse
> ... which does not necessarily keep debugging easy..
> 
  Sorry about that.  broom is fairly heavily integrated with the
tidyverse ...

> Martin :
> 
>> sessionInfo()
> R version 3.3.1 Patched (2016-07-16 r70928)
> Platform: x86_64-pc-linux-gnu (64-bit)
> Running under: Fedora 22 (Twenty Two)
> 
> locale:
>  [1] LC_CTYPE=de_CH.UTF-8       LC_NUMERIC=C              
LC_TIME=en_US.UTF-8
>  [4] LC_COLLATE=de_CH.UTF-8     LC_MONETARY=en_US.UTF-8   
LC_MESSAGES=de_CH.UTF-8
>  [7] LC_PAPER=de_CH.UTF-8       LC_NAME=C                  LC_ADDRESS=C
> [10] LC_TELEPHONE=C             LC_MEASUREMENT=de_CH.UTF-8
LC_IDENTIFICATION=C
> 
> attached base packages:
> [1] graphics  grDevices datasets  stats     utils     methods   base     
> 
> other attached packages:
> [1] broom_0.4.1     lmerTest_2.0-32 lme4_1.1-12     Matrix_1.2-3   
fortunes_1.5-3
> [6] sfsmisc_1.1-0  
> 
> loaded via a namespace (and not attached):
>  [1] Rcpp_0.12.5         compiler_3.3.1      nloptr_1.0.4       
RColorBrewer_1.1-2
>  [5] plyr_1.8.4          tools_3.3.1         rpart_4.1-10        tibble_1.1
>  [9] gtable_0.2.0        nlme_3.1-128        lattice_0.20-33    
psych_1.6.6
> [13] DBI_0.4-1           parallel_3.3.1      gridExtra_2.2.1    
stringr_1.0.0
> [17] dplyr_0.5.0         cluster_2.0.3       grid_3.3.1         
nnet_7.3-12
> [21] data.table_1.9.6    R6_2.1.2            survival_2.39-5    
foreign_0.8-66
> [25] latticeExtra_0.6-28 minqa_1.2.4         Formula_1.2-1      
tidyr_0.5.1
> [29] reshape2_1.4.1      ggplot2_2.1.0       magrittr_1.5       
Hmisc_3.17-4
> [33] scales_0.4.0        MASS_7.3-45         splines_3.3.1      
assertthat_0.1
> [37] mnormt_1.5-4        colorspace_1.2-6    stringi_1.1.1      
acepack_1.3-3.3
> [41] munsell_0.4.3       chron_2.3-47       
>>
> 
> 
>     > ## hard to show what's going on without debugging through the
function:
>     > ## the issue occurs here:
>     > ##
https://github.com/dgrtwo/broom/blob/master/R/lme4_tidiers.R#L94
> 
>     > I can't replicate this behaviour with a trivial method ...
> 
>     > silly <- function(x, ...) {
>     > UseMethod("silly")
>     > }
>     > silly.merMod <- function(object, ...) {
>     > coef(summary(object))
>     > }
>     > silly(fm1B)
>     > environment(silly) <- environment(tidy)
>     > silly(fm1B)
> 
>     > ______________________________________________
>     > R-devel at r-project.org mailing list
>     > https://stat.ethz.ch/mailman/listinfo/r-devel
>

Reasonably Related Threads

Search for more reasonably related threads

R devel - Jul 2016 - summary() dispatch puzzle

[Rd] summary() dispatch puzzle

[Rd] summary() dispatch puzzle

[Rd] summary() dispatch puzzle

Reasonably Related Threads