Holger Hoefling
2015-Feb-18 14:22 UTC
[Rd] Different serialization of functions in interactive vs. batch mode
Hi Luke, Ah - I see - thank you! This at least points me to a way on how to "fix" this. I tried setting the srcref attribute to NULL, but the hash value is still different and so is the serialization. So this looks like it is one difference, but not all of them Even if all differences were identified - it still leaves me with different behavior between interactive and batch-mode, though. I am curious as to why that is. Do you know why in interactive mode the attribute with the srcref is set, but not in batch mode? Thanks! Holger P.S. I attached the output I get when i set the attributes to NULL On Wed, Feb 18, 2015 at 3:04 PM, <luke-tierney at uiowa.edu> wrote:> Add > > attributes(testfun) > > and you will see where the two functions differ. > > luke > > > On Wed, 18 Feb 2015, Holger Hoefling wrote: > >> Hi, >> >> I posted this question to the regular help list, but it seems to be >> this is probably a question that is better addressed on r-devel. Sorry >> for the double posting. >> >> I am using hash-values to cache certain results in R. This caching >> also depends on the hash-value of the function that is being cached >> (calculated using the digest package). I noticed that computations >> that should already be cached are recomputed when switching from an >> interactive session to a BATCH session. Therefore, I wrote a test >> script >> >> library(digest) >> testfun <- function() { >> return(NULL) >> } >> testval <- "testval" >> print(digest(testfun)) >> print(serialize(testfun, connection = NULL)) >> >> and executed it once using input-redirection from a file (testFile.R) >> and once copying the code into an interactive R session. The >> hash-values of the functions differ. As digest internally relies on >> serialize, I also checked there and found that digest is not the >> reason for the discrepancy. Instead, the serialized value of the >> function already differs between the BATCH and inteactive sessions. >> >> I was wondering if someone knows if >> 1. Is this a feature or a bug? It feels like a bug to me as all the >> inputs are identical, I would expect that the output is identical as >> well. Is there something I am overlooking? >> 2. Is there a way to get consistent hash-values for functions between >> BATCH and interactive sessions. >> >> The output from the BATCH and interactive runs are below (as well as >> sessionInfo) >> >> Thank you very much for your help! >> >> Holger Hoefling >> >> --------------------------------- >> BATCH run (via input redirection): >> >> $ R --vanilla < testFile.R >> >> R version 3.1.0 (2014-04-10) -- "Spring Dance" >> Copyright (C) 2014 The R Foundation for Statistical Computing >> Platform: x86_64-unknown-linux-gnu (64-bit) >> >> R is free software and comes with ABSOLUTELY NO WARRANTY. >> You are welcome to redistribute it under certain conditions. >> Type 'license()' or 'licence()' for distribution details. >> >> R is a collaborative project with many contributors. >> Type 'contributors()' for more information and >> 'citation()' on how to cite R or R packages in publications. >> >> Type 'demo()' for some demos, 'help()' for on-line help, or >> 'help.start()' for an HTML browser interface to help. >> Type 'q()' to quit R. >> >>> library(digest) >>> testfun <- function() { >> >> + return(NULL) >> + } >>> >>> print(digest(testfun)) >> >> [1] "b03160b9250f0d5b5bcce42bd86d8e56" >>> >>> print(serialize(testfun, connection = NULL)) >> >> [1] 58 0a 00 00 00 02 00 03 01 00 00 02 03 00 00 00 04 03 00 00 00 fd 00 >> 00 00 >> [26] fe 00 00 00 06 00 00 00 01 00 04 00 09 00 00 00 01 7b 00 00 00 02 00 >> 00 00 >> [51] 06 00 00 00 01 00 04 00 09 00 00 00 06 72 65 74 75 72 6e 00 00 00 02 >> 00 00 >> [76] 00 fe 00 00 00 fe 00 00 00 fe >>> >>> sessionInfo() >> >> R version 3.1.0 (2014-04-10) >> Platform: x86_64-unknown-linux-gnu (64-bit) >> >> locale: >> [1] C >> >> attached base packages: >> [1] stats graphics grDevices utils datasets methods base >> >> other attached packages: >> [1] digest_0.6.4 >>> >>> >> >> ---------------------------------------------- >> Interactive run: >> >> $ R --vanilla >> >> R version 3.1.0 (2014-04-10) -- "Spring Dance" >> Copyright (C) 2014 The R Foundation for Statistical Computing >> Platform: x86_64-unknown-linux-gnu (64-bit) >> >> R is free software and comes with ABSOLUTELY NO WARRANTY. >> You are welcome to redistribute it under certain conditions. >> Type 'license()' or 'licence()' for distribution details. >> >> R is a collaborative project with many contributors. >> Type 'contributors()' for more information and >> 'citation()' on how to cite R or R packages in publications. >> >> Type 'demo()' for some demos, 'help()' for on-line help, or >> 'help.start()' for an HTML browser interface to help. >> Type 'q()' to quit R. >> >>> library(digest) >>> testfun <- function() { >> >> + return(NULL) >> + } >>> >>> print(digest(testfun)) >> >> [1] "fada482d2894088b079a8e56b7044862" >>> >>> print(serialize(testfun, connection = NULL)) >> >> [1] 58 0a 00 00 00 02 00 03 01 00 00 02 03 00 00 00 06 03 00 00 04 02 00 >> 00 00 >> [26] 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 03 0d 00 00 00 08 >> 00 00 >> [51] 00 01 00 00 00 0c 00 00 00 03 00 00 00 01 00 00 00 0c 00 00 00 01 00 >> 00 00 >> [76] 01 00 00 00 03 00 00 04 02 00 00 00 01 00 04 00 09 00 00 00 07 73 72 >> 63 66 >> [101] 69 6c 65 00 00 00 04 00 00 00 00 00 00 00 f2 00 00 04 02 00 00 00 01 >> 00 04 >> [126] 00 09 00 00 00 05 6c 69 6e 65 73 00 00 00 10 00 00 00 01 00 04 00 09 >> 00 00 >> [151] 00 2b 74 65 73 74 66 75 6e 20 3c 2d 20 66 75 6e 63 74 69 6f 6e 28 29 >> 20 7b >> [176] 0a 20 20 20 20 72 65 74 75 72 6e 28 4e 55 4c 4c 29 0a 7d 0a 00 00 04 >> 02 00 >> [201] 00 00 01 00 04 00 09 00 00 00 08 66 69 6c 65 6e 61 6d 65 00 00 00 10 >> 00 00 >> [226] 00 01 00 04 00 09 00 00 00 00 00 00 00 fe 00 00 00 fe 00 00 04 02 00 >> 00 00 >> [251] 01 00 04 00 09 00 00 00 05 63 6c 61 73 73 00 00 00 10 00 00 00 02 00 >> 04 00 >> [276] 09 00 00 00 0b 73 72 63 66 69 6c 65 63 6f 70 79 00 04 00 09 00 00 00 >> 07 73 >> [301] 72 63 66 69 6c 65 00 00 00 fe 00 00 04 02 00 00 06 ff 00 00 00 10 00 >> 00 00 >> [326] 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 00 fe 00 00 00 fe >> 00 00 >> [351] 00 fd 00 00 00 fe 00 00 02 06 00 00 04 02 00 00 01 ff 00 00 00 13 00 >> 00 00 >> [376] 02 00 00 03 0d 00 00 00 08 00 00 00 01 00 00 00 17 00 00 00 01 00 00 >> 00 17 >> [401] 00 00 00 17 00 00 00 17 00 00 00 01 00 00 00 01 00 00 04 02 00 00 02 >> ff 00 >> [426] 00 03 ff 00 00 04 02 00 00 06 ff 00 00 00 10 00 00 00 01 00 04 00 09 >> 00 00 >> [451] 00 06 73 72 63 72 65 66 00 00 00 fe 00 00 03 0d 00 00 00 08 00 00 00 >> 02 00 >> [476] 00 00 05 00 00 00 02 00 00 00 10 00 00 00 05 00 00 00 10 00 00 00 02 >> 00 00 >> [501] 00 02 00 00 04 02 00 00 02 ff 00 00 03 ff 00 00 04 02 00 00 06 ff 00 >> 00 00 >> [526] 10 00 00 00 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 00 fe >> 00 00 >> [551] 04 02 00 00 02 ff 00 00 03 ff 00 00 04 02 00 00 00 01 00 04 00 09 00 >> 00 00 >> [576] 0b 77 68 6f 6c 65 53 72 63 72 65 66 00 00 03 0d 00 00 00 08 00 00 00 >> 01 00 >> [601] 00 00 00 00 00 00 03 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 01 >> 00 00 >> [626] 00 03 00 00 04 02 00 00 02 ff 00 00 03 ff 00 00 04 02 00 00 06 ff 00 >> 00 00 >> [651] 10 00 00 00 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 00 fe >> 00 00 >> [676] 00 fe 00 00 00 01 00 04 00 09 00 00 00 01 7b 00 00 00 02 00 00 00 06 >> 00 00 >> [701] 00 01 00 04 00 09 00 00 00 06 72 65 74 75 72 6e 00 00 00 02 00 00 00 >> fe 00 >> [726] 00 00 fe 00 00 00 fe >>> >>> sessionInfo() >> >> R version 3.1.0 (2014-04-10) >> Platform: x86_64-unknown-linux-gnu (64-bit) >> >> locale: >> [1] C >> >> attached base packages: >> [1] stats graphics grDevices utils datasets methods base >> >> other attached packages: >> [1] digest_0.6.4 >>> >>> >> >> ______________________________________________ >> R-devel at r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel >> > > -- > Luke Tierney > Ralph E. Wareham Professor of Mathematical Sciences > University of Iowa Phone: 319-335-3386 > Department of Statistics and Fax: 319-335-3017 > Actuarial Science > 241 Schaeffer Hall email: luke-tierney at uiowa.edu > Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu-------------- next part -------------- library(digest) testfun <- function() { return(NULL) } print(digest(testfun)) print(serialize(testfun, connection = NULL)) print(attributes(testfun)) attributes(testfun)$srcref <- NULL print(attributes(testfun)) print(digest(testfun)) print(serialize(testfun, connection = NULL)) sessionInfo() ==============================================================================Interactive mode: $ R --vanilla R version 3.1.0 (2014-04-10) -- "Spring Dance" Copyright (C) 2014 The R Foundation for Statistical Computing Platform: x86_64-unknown-linux-gnu (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R.> library(digest) > testfun <- function() {+ return(NULL) + }> print(digest(testfun))[1] "fada482d2894088b079a8e56b7044862"> print(serialize(testfun, connection = NULL))[1] 58 0a 00 00 00 02 00 03 01 00 00 02 03 00 00 00 06 03 00 00 04 02 00 00 00 [26] 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 03 0d 00 00 00 08 00 00 [51] 00 01 00 00 00 0c 00 00 00 03 00 00 00 01 00 00 00 0c 00 00 00 01 00 00 00 [76] 01 00 00 00 03 00 00 04 02 00 00 00 01 00 04 00 09 00 00 00 07 73 72 63 66 [101] 69 6c 65 00 00 00 04 00 00 00 00 00 00 00 f2 00 00 04 02 00 00 00 01 00 04 [126] 00 09 00 00 00 05 6c 69 6e 65 73 00 00 00 10 00 00 00 01 00 04 00 09 00 00 [151] 00 2b 74 65 73 74 66 75 6e 20 3c 2d 20 66 75 6e 63 74 69 6f 6e 28 29 20 7b [176] 0a 20 20 20 20 72 65 74 75 72 6e 28 4e 55 4c 4c 29 0a 7d 0a 00 00 04 02 00 [201] 00 00 01 00 04 00 09 00 00 00 08 66 69 6c 65 6e 61 6d 65 00 00 00 10 00 00 [226] 00 01 00 04 00 09 00 00 00 00 00 00 00 fe 00 00 00 fe 00 00 04 02 00 00 00 [251] 01 00 04 00 09 00 00 00 05 63 6c 61 73 73 00 00 00 10 00 00 00 02 00 04 00 [276] 09 00 00 00 0b 73 72 63 66 69 6c 65 63 6f 70 79 00 04 00 09 00 00 00 07 73 [301] 72 63 66 69 6c 65 00 00 00 fe 00 00 04 02 00 00 06 ff 00 00 00 10 00 00 00 [326] 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 00 fe 00 00 00 fe 00 00 [351] 00 fd 00 00 00 fe 00 00 02 06 00 00 04 02 00 00 01 ff 00 00 00 13 00 00 00 [376] 02 00 00 03 0d 00 00 00 08 00 00 00 01 00 00 00 17 00 00 00 01 00 00 00 17 [401] 00 00 00 17 00 00 00 17 00 00 00 01 00 00 00 01 00 00 04 02 00 00 02 ff 00 [426] 00 03 ff 00 00 04 02 00 00 06 ff 00 00 00 10 00 00 00 01 00 04 00 09 00 00 [451] 00 06 73 72 63 72 65 66 00 00 00 fe 00 00 03 0d 00 00 00 08 00 00 00 02 00 [476] 00 00 05 00 00 00 02 00 00 00 10 00 00 00 05 00 00 00 10 00 00 00 02 00 00 [501] 00 02 00 00 04 02 00 00 02 ff 00 00 03 ff 00 00 04 02 00 00 06 ff 00 00 00 [526] 10 00 00 00 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 00 fe 00 00 [551] 04 02 00 00 02 ff 00 00 03 ff 00 00 04 02 00 00 00 01 00 04 00 09 00 00 00 [576] 0b 77 68 6f 6c 65 53 72 63 72 65 66 00 00 03 0d 00 00 00 08 00 00 00 01 00 [601] 00 00 00 00 00 00 03 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 01 00 00 [626] 00 03 00 00 04 02 00 00 02 ff 00 00 03 ff 00 00 04 02 00 00 06 ff 00 00 00 [651] 10 00 00 00 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 00 fe 00 00 [676] 00 fe 00 00 00 01 00 04 00 09 00 00 00 01 7b 00 00 00 02 00 00 00 06 00 00 [701] 00 01 00 04 00 09 00 00 00 06 72 65 74 75 72 6e 00 00 00 02 00 00 00 fe 00 [726] 00 00 fe 00 00 00 fe> print(attributes(testfun))$srcref function() { return(NULL) }> attributes(testfun)$srcref <- NULL > print(attributes(testfun))NULL> print(digest(testfun))[1] "87985c7cab8f65bc74dd8ac95d58ed32"> print(serialize(testfun, connection = NULL))[1] 58 0a 00 00 00 02 00 03 01 00 00 02 03 00 00 00 04 03 00 00 00 fd 00 00 00 [26] fe 00 00 02 06 00 00 04 02 00 00 00 01 00 04 00 09 00 00 00 06 73 72 63 72 [51] 65 66 00 00 00 13 00 00 00 02 00 00 03 0d 00 00 00 08 00 00 00 01 00 00 00 [76] 17 00 00 00 01 00 00 00 17 00 00 00 17 00 00 00 17 00 00 00 01 00 00 00 01 [101] 00 00 04 02 00 00 00 01 00 04 00 09 00 00 00 07 73 72 63 66 69 6c 65 00 00 [126] 00 04 00 00 00 00 00 00 00 f2 00 00 04 02 00 00 00 01 00 04 00 09 00 00 00 [151] 0d 66 69 78 65 64 4e 65 77 6c 69 6e 65 73 00 00 00 0a 00 00 00 01 00 00 00 [176] 01 00 00 04 02 00 00 00 01 00 04 00 09 00 00 00 05 6c 69 6e 65 73 00 00 00 [201] 10 00 00 00 04 00 04 00 09 00 00 00 17 74 65 73 74 66 75 6e 20 3c 2d 20 66 [226] 75 6e 63 74 69 6f 6e 28 29 20 7b 00 04 00 09 00 00 00 10 20 20 20 20 72 65 [251] 74 75 72 6e 28 4e 55 4c 4c 29 00 04 00 09 00 00 00 01 7d 00 04 00 09 00 00 [276] 00 00 00 00 04 02 00 00 00 01 00 04 00 09 00 00 00 08 66 69 6c 65 6e 61 6d [301] 65 00 00 00 10 00 00 00 01 00 04 00 09 00 00 00 00 00 00 00 fe 00 00 00 fe [326] 00 00 04 02 00 00 00 01 00 04 00 09 00 00 00 05 63 6c 61 73 73 00 00 00 10 [351] 00 00 00 02 00 04 00 09 00 00 00 0b 73 72 63 66 69 6c 65 63 6f 70 79 00 04 [376] 00 09 00 00 00 07 73 72 63 66 69 6c 65 00 00 00 fe 00 00 04 02 00 00 07 ff [401] 00 00 00 10 00 00 00 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 00 [426] fe 00 00 03 0d 00 00 00 08 00 00 00 02 00 00 00 05 00 00 00 02 00 00 00 10 [451] 00 00 00 05 00 00 00 10 00 00 00 02 00 00 00 02 00 00 04 02 00 00 02 ff 00 [476] 00 03 ff 00 00 04 02 00 00 07 ff 00 00 00 10 00 00 00 01 00 04 00 09 00 00 [501] 00 06 73 72 63 72 65 66 00 00 00 fe 00 00 04 02 00 00 02 ff 00 00 03 ff 00 [526] 00 04 02 00 00 00 01 00 04 00 09 00 00 00 0b 77 68 6f 6c 65 53 72 63 72 65 [551] 66 00 00 03 0d 00 00 00 08 00 00 00 01 00 00 00 00 00 00 00 03 00 00 00 01 [576] 00 00 00 00 00 00 00 01 00 00 00 01 00 00 00 03 00 00 04 02 00 00 02 ff 00 [601] 00 03 ff 00 00 04 02 00 00 07 ff 00 00 00 10 00 00 00 01 00 04 00 09 00 00 [626] 00 06 73 72 63 72 65 66 00 00 00 fe 00 00 00 fe 00 00 00 01 00 04 00 09 00 [651] 00 00 01 7b 00 00 00 02 00 00 00 06 00 00 00 01 00 04 00 09 00 00 00 06 72 [676] 65 74 75 72 6e 00 00 00 02 00 00 00 fe 00 00 00 fe 00 00 00 fe> sessionInfo()R version 3.1.0 (2014-04-10) Platform: x86_64-unknown-linux-gnu (64-bit) locale: [1] C attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] digest_0.6.4>==============================================================================Batch mode: $ R --vanilla < testFile.R R version 3.1.0 (2014-04-10) -- "Spring Dance" Copyright (C) 2014 The R Foundation for Statistical Computing Platform: x86_64-unknown-linux-gnu (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R.> library(digest) > testfun <- function() {+ return(NULL) + }> print(digest(testfun))[1] "b03160b9250f0d5b5bcce42bd86d8e56"> print(serialize(testfun, connection = NULL))[1] 58 0a 00 00 00 02 00 03 01 00 00 02 03 00 00 00 04 03 00 00 00 fd 00 00 00 [26] fe 00 00 00 06 00 00 00 01 00 04 00 09 00 00 00 01 7b 00 00 00 02 00 00 00 [51] 06 00 00 00 01 00 04 00 09 00 00 00 06 72 65 74 75 72 6e 00 00 00 02 00 00 [76] 00 fe 00 00 00 fe 00 00 00 fe> print(attributes(testfun))NULL> attributes(testfun)$srcref <- NULL > print(attributes(testfun))NULL> print(digest(testfun))[1] "b03160b9250f0d5b5bcce42bd86d8e56"> print(serialize(testfun, connection = NULL))[1] 58 0a 00 00 00 02 00 03 01 00 00 02 03 00 00 00 04 03 00 00 00 fd 00 00 00 [26] fe 00 00 00 06 00 00 00 01 00 04 00 09 00 00 00 01 7b 00 00 00 02 00 00 00 [51] 06 00 00 00 01 00 04 00 09 00 00 00 06 72 65 74 75 72 6e 00 00 00 02 00 00 [76] 00 fe 00 00 00 fe 00 00 00 fe> sessionInfo()R version 3.1.0 (2014-04-10) Platform: x86_64-unknown-linux-gnu (64-bit) locale: [1] C attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] digest_0.6.4>
Gabriel Becker
2015-Feb-18 14:36 UTC
[Rd] Different serialization of functions in interactive vs. batch mode
Holger, For me (see session info) using digest(as.list(f)) gets around this problem. ~G> sessionInfo()R version 3.1.0 (2014-04-10) Platform: x86_64-pc-linux-gnu (64-bit) locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] digest_0.6.8 On Wed, Feb 18, 2015 at 6:22 AM, Holger Hoefling <hhoeflin at gmail.com> wrote:> Hi Luke, > > Ah - I see - thank you! This at least points me to a way on how to > "fix" this. I tried setting the srcref attribute to NULL, but the hash > value is still different and so is the serialization. So this looks > like it is one difference, but not all of them > > Even if all differences were identified - it still leaves me with > different behavior between interactive and batch-mode, though. I am > curious as to why that is. Do you know why in interactive mode the > attribute with the srcref is set, but not in batch mode? > > Thanks! > > Holger > > P.S. I attached the output I get when i set the attributes to NULL > > > On Wed, Feb 18, 2015 at 3:04 PM, <luke-tierney at uiowa.edu> wrote: > > Add > > > > attributes(testfun) > > > > and you will see where the two functions differ. > > > > luke > > > > > > On Wed, 18 Feb 2015, Holger Hoefling wrote: > > > >> Hi, > >> > >> I posted this question to the regular help list, but it seems to be > >> this is probably a question that is better addressed on r-devel. Sorry > >> for the double posting. > >> > >> I am using hash-values to cache certain results in R. This caching > >> also depends on the hash-value of the function that is being cached > >> (calculated using the digest package). I noticed that computations > >> that should already be cached are recomputed when switching from an > >> interactive session to a BATCH session. Therefore, I wrote a test > >> script > >> > >> library(digest) > >> testfun <- function() { > >> return(NULL) > >> } > >> testval <- "testval" > >> print(digest(testfun)) > >> print(serialize(testfun, connection = NULL)) > >> > >> and executed it once using input-redirection from a file (testFile.R) > >> and once copying the code into an interactive R session. The > >> hash-values of the functions differ. As digest internally relies on > >> serialize, I also checked there and found that digest is not the > >> reason for the discrepancy. Instead, the serialized value of the > >> function already differs between the BATCH and inteactive sessions. > >> > >> I was wondering if someone knows if > >> 1. Is this a feature or a bug? It feels like a bug to me as all the > >> inputs are identical, I would expect that the output is identical as > >> well. Is there something I am overlooking? > >> 2. Is there a way to get consistent hash-values for functions between > >> BATCH and interactive sessions. > >> > >> The output from the BATCH and interactive runs are below (as well as > >> sessionInfo) > >> > >> Thank you very much for your help! > >> > >> Holger Hoefling > >> > >> --------------------------------- > >> BATCH run (via input redirection): > >> > >> $ R --vanilla < testFile.R > >> > >> R version 3.1.0 (2014-04-10) -- "Spring Dance" > >> Copyright (C) 2014 The R Foundation for Statistical Computing > >> Platform: x86_64-unknown-linux-gnu (64-bit) > >> > >> R is free software and comes with ABSOLUTELY NO WARRANTY. > >> You are welcome to redistribute it under certain conditions. > >> Type 'license()' or 'licence()' for distribution details. > >> > >> R is a collaborative project with many contributors. > >> Type 'contributors()' for more information and > >> 'citation()' on how to cite R or R packages in publications. > >> > >> Type 'demo()' for some demos, 'help()' for on-line help, or > >> 'help.start()' for an HTML browser interface to help. > >> Type 'q()' to quit R. > >> > >>> library(digest) > >>> testfun <- function() { > >> > >> + return(NULL) > >> + } > >>> > >>> print(digest(testfun)) > >> > >> [1] "b03160b9250f0d5b5bcce42bd86d8e56" > >>> > >>> print(serialize(testfun, connection = NULL)) > >> > >> [1] 58 0a 00 00 00 02 00 03 01 00 00 02 03 00 00 00 04 03 00 00 00 fd 00 > >> 00 00 > >> [26] fe 00 00 00 06 00 00 00 01 00 04 00 09 00 00 00 01 7b 00 00 00 02 > 00 > >> 00 00 > >> [51] 06 00 00 00 01 00 04 00 09 00 00 00 06 72 65 74 75 72 6e 00 00 00 > 02 > >> 00 00 > >> [76] 00 fe 00 00 00 fe 00 00 00 fe > >>> > >>> sessionInfo() > >> > >> R version 3.1.0 (2014-04-10) > >> Platform: x86_64-unknown-linux-gnu (64-bit) > >> > >> locale: > >> [1] C > >> > >> attached base packages: > >> [1] stats graphics grDevices utils datasets methods base > >> > >> other attached packages: > >> [1] digest_0.6.4 > >>> > >>> > >> > >> ---------------------------------------------- > >> Interactive run: > >> > >> $ R --vanilla > >> > >> R version 3.1.0 (2014-04-10) -- "Spring Dance" > >> Copyright (C) 2014 The R Foundation for Statistical Computing > >> Platform: x86_64-unknown-linux-gnu (64-bit) > >> > >> R is free software and comes with ABSOLUTELY NO WARRANTY. > >> You are welcome to redistribute it under certain conditions. > >> Type 'license()' or 'licence()' for distribution details. > >> > >> R is a collaborative project with many contributors. > >> Type 'contributors()' for more information and > >> 'citation()' on how to cite R or R packages in publications. > >> > >> Type 'demo()' for some demos, 'help()' for on-line help, or > >> 'help.start()' for an HTML browser interface to help. > >> Type 'q()' to quit R. > >> > >>> library(digest) > >>> testfun <- function() { > >> > >> + return(NULL) > >> + } > >>> > >>> print(digest(testfun)) > >> > >> [1] "fada482d2894088b079a8e56b7044862" > >>> > >>> print(serialize(testfun, connection = NULL)) > >> > >> [1] 58 0a 00 00 00 02 00 03 01 00 00 02 03 00 00 00 06 03 00 00 04 02 > 00 > >> 00 00 > >> [26] 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 03 0d 00 00 00 > 08 > >> 00 00 > >> [51] 00 01 00 00 00 0c 00 00 00 03 00 00 00 01 00 00 00 0c 00 00 00 01 > 00 > >> 00 00 > >> [76] 01 00 00 00 03 00 00 04 02 00 00 00 01 00 04 00 09 00 00 00 07 73 > 72 > >> 63 66 > >> [101] 69 6c 65 00 00 00 04 00 00 00 00 00 00 00 f2 00 00 04 02 00 00 00 > 01 > >> 00 04 > >> [126] 00 09 00 00 00 05 6c 69 6e 65 73 00 00 00 10 00 00 00 01 00 04 00 > 09 > >> 00 00 > >> [151] 00 2b 74 65 73 74 66 75 6e 20 3c 2d 20 66 75 6e 63 74 69 6f 6e 28 > 29 > >> 20 7b > >> [176] 0a 20 20 20 20 72 65 74 75 72 6e 28 4e 55 4c 4c 29 0a 7d 0a 00 00 > 04 > >> 02 00 > >> [201] 00 00 01 00 04 00 09 00 00 00 08 66 69 6c 65 6e 61 6d 65 00 00 00 > 10 > >> 00 00 > >> [226] 00 01 00 04 00 09 00 00 00 00 00 00 00 fe 00 00 00 fe 00 00 04 02 > 00 > >> 00 00 > >> [251] 01 00 04 00 09 00 00 00 05 63 6c 61 73 73 00 00 00 10 00 00 00 02 > 00 > >> 04 00 > >> [276] 09 00 00 00 0b 73 72 63 66 69 6c 65 63 6f 70 79 00 04 00 09 00 00 > 00 > >> 07 73 > >> [301] 72 63 66 69 6c 65 00 00 00 fe 00 00 04 02 00 00 06 ff 00 00 00 10 > 00 > >> 00 00 > >> [326] 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 00 fe 00 00 00 > fe > >> 00 00 > >> [351] 00 fd 00 00 00 fe 00 00 02 06 00 00 04 02 00 00 01 ff 00 00 00 13 > 00 > >> 00 00 > >> [376] 02 00 00 03 0d 00 00 00 08 00 00 00 01 00 00 00 17 00 00 00 01 00 > 00 > >> 00 17 > >> [401] 00 00 00 17 00 00 00 17 00 00 00 01 00 00 00 01 00 00 04 02 00 00 > 02 > >> ff 00 > >> [426] 00 03 ff 00 00 04 02 00 00 06 ff 00 00 00 10 00 00 00 01 00 04 00 > 09 > >> 00 00 > >> [451] 00 06 73 72 63 72 65 66 00 00 00 fe 00 00 03 0d 00 00 00 08 00 00 > 00 > >> 02 00 > >> [476] 00 00 05 00 00 00 02 00 00 00 10 00 00 00 05 00 00 00 10 00 00 00 > 02 > >> 00 00 > >> [501] 00 02 00 00 04 02 00 00 02 ff 00 00 03 ff 00 00 04 02 00 00 06 ff > 00 > >> 00 00 > >> [526] 10 00 00 00 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 00 > fe > >> 00 00 > >> [551] 04 02 00 00 02 ff 00 00 03 ff 00 00 04 02 00 00 00 01 00 04 00 09 > 00 > >> 00 00 > >> [576] 0b 77 68 6f 6c 65 53 72 63 72 65 66 00 00 03 0d 00 00 00 08 00 00 > 00 > >> 01 00 > >> [601] 00 00 00 00 00 00 03 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 > 01 > >> 00 00 > >> [626] 00 03 00 00 04 02 00 00 02 ff 00 00 03 ff 00 00 04 02 00 00 06 ff > 00 > >> 00 00 > >> [651] 10 00 00 00 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 00 > fe > >> 00 00 > >> [676] 00 fe 00 00 00 01 00 04 00 09 00 00 00 01 7b 00 00 00 02 00 00 00 > 06 > >> 00 00 > >> [701] 00 01 00 04 00 09 00 00 00 06 72 65 74 75 72 6e 00 00 00 02 00 00 > 00 > >> fe 00 > >> [726] 00 00 fe 00 00 00 fe > >>> > >>> sessionInfo() > >> > >> R version 3.1.0 (2014-04-10) > >> Platform: x86_64-unknown-linux-gnu (64-bit) > >> > >> locale: > >> [1] C > >> > >> attached base packages: > >> [1] stats graphics grDevices utils datasets methods base > >> > >> other attached packages: > >> [1] digest_0.6.4 > >>> > >>> > >> > >> ______________________________________________ > >> R-devel at r-project.org mailing list > >> https://stat.ethz.ch/mailman/listinfo/r-devel > >> > > > > -- > > Luke Tierney > > Ralph E. Wareham Professor of Mathematical Sciences > > University of Iowa Phone: 319-335-3386 > > Department of Statistics and Fax: 319-335-3017 > > Actuarial Science > > 241 Schaeffer Hall email: luke-tierney at uiowa.edu > > Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > >-- Gabriel Becker, PhD Computational Biologist Bioinformatics and Computational Biology Genentech, Inc. [[alternative HTML version deleted]]
Holger Hoefling
2015-Feb-18 14:43 UTC
[Rd] Different serialization of functions in interactive vs. batch mode
Hi Gabriel, thanks for your reply - it does solve the problem of my toy function, but does come with some other problems though. a) as.list(f)[[1]] yields an expression, not a function. In order to go the route you are suggesting, I would more likely use "deparse" in order to get the original back b) and more seriously - as.list strips the environment of the function (and thus associated information) as well as information about parent environments. For the execution of a function, this would however be crucial. This is also why a pure "deparse" alone in my opinion would not work. Thanks Holger On Wed, Feb 18, 2015 at 3:36 PM, Gabriel Becker <gmbecker at ucdavis.edu> wrote:> Holger, > > For me (see session info) using > > digest(as.list(f)) > > gets around this problem. > > ~G > >> sessionInfo() > R version 3.1.0 (2014-04-10) > Platform: x86_64-pc-linux-gnu (64-bit) > > locale: > [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C > [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 > [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 > [7] LC_PAPER=en_US.UTF-8 LC_NAME=C > [9] LC_ADDRESS=C LC_TELEPHONE=C > [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > other attached packages: > [1] digest_0.6.8 > > > > > On Wed, Feb 18, 2015 at 6:22 AM, Holger Hoefling <hhoeflin at gmail.com> wrote: >> >> Hi Luke, >> >> Ah - I see - thank you! This at least points me to a way on how to >> "fix" this. I tried setting the srcref attribute to NULL, but the hash >> value is still different and so is the serialization. So this looks >> like it is one difference, but not all of them >> >> Even if all differences were identified - it still leaves me with >> different behavior between interactive and batch-mode, though. I am >> curious as to why that is. Do you know why in interactive mode the >> attribute with the srcref is set, but not in batch mode? >> >> Thanks! >> >> Holger >> >> P.S. I attached the output I get when i set the attributes to NULL >> >> >> On Wed, Feb 18, 2015 at 3:04 PM, <luke-tierney at uiowa.edu> wrote: >> > Add >> > >> > attributes(testfun) >> > >> > and you will see where the two functions differ. >> > >> > luke >> > >> > >> > On Wed, 18 Feb 2015, Holger Hoefling wrote: >> > >> >> Hi, >> >> >> >> I posted this question to the regular help list, but it seems to be >> >> this is probably a question that is better addressed on r-devel. Sorry >> >> for the double posting. >> >> >> >> I am using hash-values to cache certain results in R. This caching >> >> also depends on the hash-value of the function that is being cached >> >> (calculated using the digest package). I noticed that computations >> >> that should already be cached are recomputed when switching from an >> >> interactive session to a BATCH session. Therefore, I wrote a test >> >> script >> >> >> >> library(digest) >> >> testfun <- function() { >> >> return(NULL) >> >> } >> >> testval <- "testval" >> >> print(digest(testfun)) >> >> print(serialize(testfun, connection = NULL)) >> >> >> >> and executed it once using input-redirection from a file (testFile.R) >> >> and once copying the code into an interactive R session. The >> >> hash-values of the functions differ. As digest internally relies on >> >> serialize, I also checked there and found that digest is not the >> >> reason for the discrepancy. Instead, the serialized value of the >> >> function already differs between the BATCH and inteactive sessions. >> >> >> >> I was wondering if someone knows if >> >> 1. Is this a feature or a bug? It feels like a bug to me as all the >> >> inputs are identical, I would expect that the output is identical as >> >> well. Is there something I am overlooking? >> >> 2. Is there a way to get consistent hash-values for functions between >> >> BATCH and interactive sessions. >> >> >> >> The output from the BATCH and interactive runs are below (as well as >> >> sessionInfo) >> >> >> >> Thank you very much for your help! >> >> >> >> Holger Hoefling >> >> >> >> --------------------------------- >> >> BATCH run (via input redirection): >> >> >> >> $ R --vanilla < testFile.R >> >> >> >> R version 3.1.0 (2014-04-10) -- "Spring Dance" >> >> Copyright (C) 2014 The R Foundation for Statistical Computing >> >> Platform: x86_64-unknown-linux-gnu (64-bit) >> >> >> >> R is free software and comes with ABSOLUTELY NO WARRANTY. >> >> You are welcome to redistribute it under certain conditions. >> >> Type 'license()' or 'licence()' for distribution details. >> >> >> >> R is a collaborative project with many contributors. >> >> Type 'contributors()' for more information and >> >> 'citation()' on how to cite R or R packages in publications. >> >> >> >> Type 'demo()' for some demos, 'help()' for on-line help, or >> >> 'help.start()' for an HTML browser interface to help. >> >> Type 'q()' to quit R. >> >> >> >>> library(digest) >> >>> testfun <- function() { >> >> >> >> + return(NULL) >> >> + } >> >>> >> >>> print(digest(testfun)) >> >> >> >> [1] "b03160b9250f0d5b5bcce42bd86d8e56" >> >>> >> >>> print(serialize(testfun, connection = NULL)) >> >> >> >> [1] 58 0a 00 00 00 02 00 03 01 00 00 02 03 00 00 00 04 03 00 00 00 fd >> >> 00 >> >> 00 00 >> >> [26] fe 00 00 00 06 00 00 00 01 00 04 00 09 00 00 00 01 7b 00 00 00 02 >> >> 00 >> >> 00 00 >> >> [51] 06 00 00 00 01 00 04 00 09 00 00 00 06 72 65 74 75 72 6e 00 00 00 >> >> 02 >> >> 00 00 >> >> [76] 00 fe 00 00 00 fe 00 00 00 fe >> >>> >> >>> sessionInfo() >> >> >> >> R version 3.1.0 (2014-04-10) >> >> Platform: x86_64-unknown-linux-gnu (64-bit) >> >> >> >> locale: >> >> [1] C >> >> >> >> attached base packages: >> >> [1] stats graphics grDevices utils datasets methods base >> >> >> >> other attached packages: >> >> [1] digest_0.6.4 >> >>> >> >>> >> >> >> >> ---------------------------------------------- >> >> Interactive run: >> >> >> >> $ R --vanilla >> >> >> >> R version 3.1.0 (2014-04-10) -- "Spring Dance" >> >> Copyright (C) 2014 The R Foundation for Statistical Computing >> >> Platform: x86_64-unknown-linux-gnu (64-bit) >> >> >> >> R is free software and comes with ABSOLUTELY NO WARRANTY. >> >> You are welcome to redistribute it under certain conditions. >> >> Type 'license()' or 'licence()' for distribution details. >> >> >> >> R is a collaborative project with many contributors. >> >> Type 'contributors()' for more information and >> >> 'citation()' on how to cite R or R packages in publications. >> >> >> >> Type 'demo()' for some demos, 'help()' for on-line help, or >> >> 'help.start()' for an HTML browser interface to help. >> >> Type 'q()' to quit R. >> >> >> >>> library(digest) >> >>> testfun <- function() { >> >> >> >> + return(NULL) >> >> + } >> >>> >> >>> print(digest(testfun)) >> >> >> >> [1] "fada482d2894088b079a8e56b7044862" >> >>> >> >>> print(serialize(testfun, connection = NULL)) >> >> >> >> [1] 58 0a 00 00 00 02 00 03 01 00 00 02 03 00 00 00 06 03 00 00 04 02 >> >> 00 >> >> 00 00 >> >> [26] 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 03 0d 00 00 00 >> >> 08 >> >> 00 00 >> >> [51] 00 01 00 00 00 0c 00 00 00 03 00 00 00 01 00 00 00 0c 00 00 00 01 >> >> 00 >> >> 00 00 >> >> [76] 01 00 00 00 03 00 00 04 02 00 00 00 01 00 04 00 09 00 00 00 07 73 >> >> 72 >> >> 63 66 >> >> [101] 69 6c 65 00 00 00 04 00 00 00 00 00 00 00 f2 00 00 04 02 00 00 00 >> >> 01 >> >> 00 04 >> >> [126] 00 09 00 00 00 05 6c 69 6e 65 73 00 00 00 10 00 00 00 01 00 04 00 >> >> 09 >> >> 00 00 >> >> [151] 00 2b 74 65 73 74 66 75 6e 20 3c 2d 20 66 75 6e 63 74 69 6f 6e 28 >> >> 29 >> >> 20 7b >> >> [176] 0a 20 20 20 20 72 65 74 75 72 6e 28 4e 55 4c 4c 29 0a 7d 0a 00 00 >> >> 04 >> >> 02 00 >> >> [201] 00 00 01 00 04 00 09 00 00 00 08 66 69 6c 65 6e 61 6d 65 00 00 00 >> >> 10 >> >> 00 00 >> >> [226] 00 01 00 04 00 09 00 00 00 00 00 00 00 fe 00 00 00 fe 00 00 04 02 >> >> 00 >> >> 00 00 >> >> [251] 01 00 04 00 09 00 00 00 05 63 6c 61 73 73 00 00 00 10 00 00 00 02 >> >> 00 >> >> 04 00 >> >> [276] 09 00 00 00 0b 73 72 63 66 69 6c 65 63 6f 70 79 00 04 00 09 00 00 >> >> 00 >> >> 07 73 >> >> [301] 72 63 66 69 6c 65 00 00 00 fe 00 00 04 02 00 00 06 ff 00 00 00 10 >> >> 00 >> >> 00 00 >> >> [326] 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 00 fe 00 00 00 >> >> fe >> >> 00 00 >> >> [351] 00 fd 00 00 00 fe 00 00 02 06 00 00 04 02 00 00 01 ff 00 00 00 13 >> >> 00 >> >> 00 00 >> >> [376] 02 00 00 03 0d 00 00 00 08 00 00 00 01 00 00 00 17 00 00 00 01 00 >> >> 00 >> >> 00 17 >> >> [401] 00 00 00 17 00 00 00 17 00 00 00 01 00 00 00 01 00 00 04 02 00 00 >> >> 02 >> >> ff 00 >> >> [426] 00 03 ff 00 00 04 02 00 00 06 ff 00 00 00 10 00 00 00 01 00 04 00 >> >> 09 >> >> 00 00 >> >> [451] 00 06 73 72 63 72 65 66 00 00 00 fe 00 00 03 0d 00 00 00 08 00 00 >> >> 00 >> >> 02 00 >> >> [476] 00 00 05 00 00 00 02 00 00 00 10 00 00 00 05 00 00 00 10 00 00 00 >> >> 02 >> >> 00 00 >> >> [501] 00 02 00 00 04 02 00 00 02 ff 00 00 03 ff 00 00 04 02 00 00 06 ff >> >> 00 >> >> 00 00 >> >> [526] 10 00 00 00 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 00 >> >> fe >> >> 00 00 >> >> [551] 04 02 00 00 02 ff 00 00 03 ff 00 00 04 02 00 00 00 01 00 04 00 09 >> >> 00 >> >> 00 00 >> >> [576] 0b 77 68 6f 6c 65 53 72 63 72 65 66 00 00 03 0d 00 00 00 08 00 00 >> >> 00 >> >> 01 00 >> >> [601] 00 00 00 00 00 00 03 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 >> >> 01 >> >> 00 00 >> >> [626] 00 03 00 00 04 02 00 00 02 ff 00 00 03 ff 00 00 04 02 00 00 06 ff >> >> 00 >> >> 00 00 >> >> [651] 10 00 00 00 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 00 >> >> fe >> >> 00 00 >> >> [676] 00 fe 00 00 00 01 00 04 00 09 00 00 00 01 7b 00 00 00 02 00 00 00 >> >> 06 >> >> 00 00 >> >> [701] 00 01 00 04 00 09 00 00 00 06 72 65 74 75 72 6e 00 00 00 02 00 00 >> >> 00 >> >> fe 00 >> >> [726] 00 00 fe 00 00 00 fe >> >>> >> >>> sessionInfo() >> >> >> >> R version 3.1.0 (2014-04-10) >> >> Platform: x86_64-unknown-linux-gnu (64-bit) >> >> >> >> locale: >> >> [1] C >> >> >> >> attached base packages: >> >> [1] stats graphics grDevices utils datasets methods base >> >> >> >> other attached packages: >> >> [1] digest_0.6.4 >> >>> >> >>> >> >> >> >> ______________________________________________ >> >> R-devel at r-project.org mailing list >> >> https://stat.ethz.ch/mailman/listinfo/r-devel >> >> >> > >> > -- >> > Luke Tierney >> > Ralph E. Wareham Professor of Mathematical Sciences >> > University of Iowa Phone: 319-335-3386 >> > Department of Statistics and Fax: 319-335-3017 >> > Actuarial Science >> > 241 Schaeffer Hall email: luke-tierney at uiowa.edu >> > Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu >> >> ______________________________________________ >> R-devel at r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel >> > > > > -- > Gabriel Becker, PhD > Computational Biologist > Bioinformatics and Computational Biology > Genentech, Inc.
luke-tierney at uiowa.edu
2015-Feb-18 14:56 UTC
[Rd] Different serialization of functions in interactive vs. batch mode
On Wed, 18 Feb 2015, Holger Hoefling wrote:> Hi Luke, > > Ah - I see - thank you! This at least points me to a way on how to > "fix" this. I tried setting the srcref attribute to NULL, but the hash > value is still different and so is the serialization. So this looks > like it is one difference, but not all of themwhen source attributes are attached they are attached to calls within the body as well, so > attributes(body(testfun)) $srcref $srcref[[1]] ...> Even if all differences were identified - it still leaves me with > different behavior between interactive and batch-mode, though. I am > curious as to why that is. Do you know why in interactive mode the > attribute with the srcref is set, but not in batch mode?I can't speak to why the default for the keep.source option is different in the two cases, but you can set it to be the same: in interactive mode > options(keep.source = FALSE) > testfun <- function() {return(NULL)} > digest(testfun) [1] "b03160b9250f0d5b5bcce42bd86d8e56" Or use options(keep.source = TRUE) in the other case. Best, luke> > Thanks! > > Holger > > P.S. I attached the output I get when i set the attributes to NULL > > > On Wed, Feb 18, 2015 at 3:04 PM, <luke-tierney at uiowa.edu> wrote: >> Add >> >> attributes(testfun) >> >> and you will see where the two functions differ. >> >> luke >> >> >> On Wed, 18 Feb 2015, Holger Hoefling wrote: >> >>> Hi, >>> >>> I posted this question to the regular help list, but it seems to be >>> this is probably a question that is better addressed on r-devel. Sorry >>> for the double posting. >>> >>> I am using hash-values to cache certain results in R. This caching >>> also depends on the hash-value of the function that is being cached >>> (calculated using the digest package). I noticed that computations >>> that should already be cached are recomputed when switching from an >>> interactive session to a BATCH session. Therefore, I wrote a test >>> script >>> >>> library(digest) >>> testfun <- function() { >>> return(NULL) >>> } >>> testval <- "testval" >>> print(digest(testfun)) >>> print(serialize(testfun, connection = NULL)) >>> >>> and executed it once using input-redirection from a file (testFile.R) >>> and once copying the code into an interactive R session. The >>> hash-values of the functions differ. As digest internally relies on >>> serialize, I also checked there and found that digest is not the >>> reason for the discrepancy. Instead, the serialized value of the >>> function already differs between the BATCH and inteactive sessions. >>> >>> I was wondering if someone knows if >>> 1. Is this a feature or a bug? It feels like a bug to me as all the >>> inputs are identical, I would expect that the output is identical as >>> well. Is there something I am overlooking? >>> 2. Is there a way to get consistent hash-values for functions between >>> BATCH and interactive sessions. >>> >>> The output from the BATCH and interactive runs are below (as well as >>> sessionInfo) >>> >>> Thank you very much for your help! >>> >>> Holger Hoefling >>> >>> --------------------------------- >>> BATCH run (via input redirection): >>> >>> $ R --vanilla < testFile.R >>> >>> R version 3.1.0 (2014-04-10) -- "Spring Dance" >>> Copyright (C) 2014 The R Foundation for Statistical Computing >>> Platform: x86_64-unknown-linux-gnu (64-bit) >>> >>> R is free software and comes with ABSOLUTELY NO WARRANTY. >>> You are welcome to redistribute it under certain conditions. >>> Type 'license()' or 'licence()' for distribution details. >>> >>> R is a collaborative project with many contributors. >>> Type 'contributors()' for more information and >>> 'citation()' on how to cite R or R packages in publications. >>> >>> Type 'demo()' for some demos, 'help()' for on-line help, or >>> 'help.start()' for an HTML browser interface to help. >>> Type 'q()' to quit R. >>> >>>> library(digest) >>>> testfun <- function() { >>> >>> + return(NULL) >>> + } >>>> >>>> print(digest(testfun)) >>> >>> [1] "b03160b9250f0d5b5bcce42bd86d8e56" >>>> >>>> print(serialize(testfun, connection = NULL)) >>> >>> [1] 58 0a 00 00 00 02 00 03 01 00 00 02 03 00 00 00 04 03 00 00 00 fd 00 >>> 00 00 >>> [26] fe 00 00 00 06 00 00 00 01 00 04 00 09 00 00 00 01 7b 00 00 00 02 00 >>> 00 00 >>> [51] 06 00 00 00 01 00 04 00 09 00 00 00 06 72 65 74 75 72 6e 00 00 00 02 >>> 00 00 >>> [76] 00 fe 00 00 00 fe 00 00 00 fe >>>> >>>> sessionInfo() >>> >>> R version 3.1.0 (2014-04-10) >>> Platform: x86_64-unknown-linux-gnu (64-bit) >>> >>> locale: >>> [1] C >>> >>> attached base packages: >>> [1] stats graphics grDevices utils datasets methods base >>> >>> other attached packages: >>> [1] digest_0.6.4 >>>> >>>> >>> >>> ---------------------------------------------- >>> Interactive run: >>> >>> $ R --vanilla >>> >>> R version 3.1.0 (2014-04-10) -- "Spring Dance" >>> Copyright (C) 2014 The R Foundation for Statistical Computing >>> Platform: x86_64-unknown-linux-gnu (64-bit) >>> >>> R is free software and comes with ABSOLUTELY NO WARRANTY. >>> You are welcome to redistribute it under certain conditions. >>> Type 'license()' or 'licence()' for distribution details. >>> >>> R is a collaborative project with many contributors. >>> Type 'contributors()' for more information and >>> 'citation()' on how to cite R or R packages in publications. >>> >>> Type 'demo()' for some demos, 'help()' for on-line help, or >>> 'help.start()' for an HTML browser interface to help. >>> Type 'q()' to quit R. >>> >>>> library(digest) >>>> testfun <- function() { >>> >>> + return(NULL) >>> + } >>>> >>>> print(digest(testfun)) >>> >>> [1] "fada482d2894088b079a8e56b7044862" >>>> >>>> print(serialize(testfun, connection = NULL)) >>> >>> [1] 58 0a 00 00 00 02 00 03 01 00 00 02 03 00 00 00 06 03 00 00 04 02 00 >>> 00 00 >>> [26] 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 03 0d 00 00 00 08 >>> 00 00 >>> [51] 00 01 00 00 00 0c 00 00 00 03 00 00 00 01 00 00 00 0c 00 00 00 01 00 >>> 00 00 >>> [76] 01 00 00 00 03 00 00 04 02 00 00 00 01 00 04 00 09 00 00 00 07 73 72 >>> 63 66 >>> [101] 69 6c 65 00 00 00 04 00 00 00 00 00 00 00 f2 00 00 04 02 00 00 00 01 >>> 00 04 >>> [126] 00 09 00 00 00 05 6c 69 6e 65 73 00 00 00 10 00 00 00 01 00 04 00 09 >>> 00 00 >>> [151] 00 2b 74 65 73 74 66 75 6e 20 3c 2d 20 66 75 6e 63 74 69 6f 6e 28 29 >>> 20 7b >>> [176] 0a 20 20 20 20 72 65 74 75 72 6e 28 4e 55 4c 4c 29 0a 7d 0a 00 00 04 >>> 02 00 >>> [201] 00 00 01 00 04 00 09 00 00 00 08 66 69 6c 65 6e 61 6d 65 00 00 00 10 >>> 00 00 >>> [226] 00 01 00 04 00 09 00 00 00 00 00 00 00 fe 00 00 00 fe 00 00 04 02 00 >>> 00 00 >>> [251] 01 00 04 00 09 00 00 00 05 63 6c 61 73 73 00 00 00 10 00 00 00 02 00 >>> 04 00 >>> [276] 09 00 00 00 0b 73 72 63 66 69 6c 65 63 6f 70 79 00 04 00 09 00 00 00 >>> 07 73 >>> [301] 72 63 66 69 6c 65 00 00 00 fe 00 00 04 02 00 00 06 ff 00 00 00 10 00 >>> 00 00 >>> [326] 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 00 fe 00 00 00 fe >>> 00 00 >>> [351] 00 fd 00 00 00 fe 00 00 02 06 00 00 04 02 00 00 01 ff 00 00 00 13 00 >>> 00 00 >>> [376] 02 00 00 03 0d 00 00 00 08 00 00 00 01 00 00 00 17 00 00 00 01 00 00 >>> 00 17 >>> [401] 00 00 00 17 00 00 00 17 00 00 00 01 00 00 00 01 00 00 04 02 00 00 02 >>> ff 00 >>> [426] 00 03 ff 00 00 04 02 00 00 06 ff 00 00 00 10 00 00 00 01 00 04 00 09 >>> 00 00 >>> [451] 00 06 73 72 63 72 65 66 00 00 00 fe 00 00 03 0d 00 00 00 08 00 00 00 >>> 02 00 >>> [476] 00 00 05 00 00 00 02 00 00 00 10 00 00 00 05 00 00 00 10 00 00 00 02 >>> 00 00 >>> [501] 00 02 00 00 04 02 00 00 02 ff 00 00 03 ff 00 00 04 02 00 00 06 ff 00 >>> 00 00 >>> [526] 10 00 00 00 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 00 fe >>> 00 00 >>> [551] 04 02 00 00 02 ff 00 00 03 ff 00 00 04 02 00 00 00 01 00 04 00 09 00 >>> 00 00 >>> [576] 0b 77 68 6f 6c 65 53 72 63 72 65 66 00 00 03 0d 00 00 00 08 00 00 00 >>> 01 00 >>> [601] 00 00 00 00 00 00 03 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 01 >>> 00 00 >>> [626] 00 03 00 00 04 02 00 00 02 ff 00 00 03 ff 00 00 04 02 00 00 06 ff 00 >>> 00 00 >>> [651] 10 00 00 00 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 00 fe >>> 00 00 >>> [676] 00 fe 00 00 00 01 00 04 00 09 00 00 00 01 7b 00 00 00 02 00 00 00 06 >>> 00 00 >>> [701] 00 01 00 04 00 09 00 00 00 06 72 65 74 75 72 6e 00 00 00 02 00 00 00 >>> fe 00 >>> [726] 00 00 fe 00 00 00 fe >>>> >>>> sessionInfo() >>> >>> R version 3.1.0 (2014-04-10) >>> Platform: x86_64-unknown-linux-gnu (64-bit) >>> >>> locale: >>> [1] C >>> >>> attached base packages: >>> [1] stats graphics grDevices utils datasets methods base >>> >>> other attached packages: >>> [1] digest_0.6.4 >>>> >>>> >>> >>> ______________________________________________ >>> R-devel at r-project.org mailing list >>> https://stat.ethz.ch/mailman/listinfo/r-devel >>> >> >> -- >> Luke Tierney >> Ralph E. Wareham Professor of Mathematical Sciences >> University of Iowa Phone: 319-335-3386 >> Department of Statistics and Fax: 319-335-3017 >> Actuarial Science >> 241 Schaeffer Hall email: luke-tierney at uiowa.edu >> Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu >-- Luke Tierney Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics and Fax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: luke-tierney at uiowa.edu Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu
Holger Hoefling
2015-Feb-18 15:02 UTC
[Rd] Different serialization of functions in interactive vs. batch mode
Luke, thank you so much - that solves my problem! Holger On Wed, Feb 18, 2015 at 3:56 PM, <luke-tierney at uiowa.edu> wrote:> On Wed, 18 Feb 2015, Holger Hoefling wrote: > >> Hi Luke, >> >> Ah - I see - thank you! This at least points me to a way on how to >> "fix" this. I tried setting the srcref attribute to NULL, but the hash >> value is still different and so is the serialization. So this looks >> like it is one difference, but not all of them > > > when source attributes are attached they are attached to calls within > the body as well, so > > > attributes(body(testfun)) > $srcref > $srcref[[1]] > ... > >> Even if all differences were identified - it still leaves me with >> different behavior between interactive and batch-mode, though. I am >> curious as to why that is. Do you know why in interactive mode the >> attribute with the srcref is set, but not in batch mode? > > > I can't speak to why the default for the keep.source option is > different in the two cases, but you can set it to be the same: in > interactive mode > > > options(keep.source = FALSE) > > testfun <- function() {return(NULL)} > > digest(testfun) > [1] "b03160b9250f0d5b5bcce42bd86d8e56" > > Or use options(keep.source = TRUE) in the other case. > > Best, > > luke > > >> >> Thanks! >> >> Holger >> >> P.S. I attached the output I get when i set the attributes to NULL >> >> >> On Wed, Feb 18, 2015 at 3:04 PM, <luke-tierney at uiowa.edu> wrote: >>> >>> Add >>> >>> attributes(testfun) >>> >>> and you will see where the two functions differ. >>> >>> luke >>> >>> >>> On Wed, 18 Feb 2015, Holger Hoefling wrote: >>> >>>> Hi, >>>> >>>> I posted this question to the regular help list, but it seems to be >>>> this is probably a question that is better addressed on r-devel. Sorry >>>> for the double posting. >>>> >>>> I am using hash-values to cache certain results in R. This caching >>>> also depends on the hash-value of the function that is being cached >>>> (calculated using the digest package). I noticed that computations >>>> that should already be cached are recomputed when switching from an >>>> interactive session to a BATCH session. Therefore, I wrote a test >>>> script >>>> >>>> library(digest) >>>> testfun <- function() { >>>> return(NULL) >>>> } >>>> testval <- "testval" >>>> print(digest(testfun)) >>>> print(serialize(testfun, connection = NULL)) >>>> >>>> and executed it once using input-redirection from a file (testFile.R) >>>> and once copying the code into an interactive R session. The >>>> hash-values of the functions differ. As digest internally relies on >>>> serialize, I also checked there and found that digest is not the >>>> reason for the discrepancy. Instead, the serialized value of the >>>> function already differs between the BATCH and inteactive sessions. >>>> >>>> I was wondering if someone knows if >>>> 1. Is this a feature or a bug? It feels like a bug to me as all the >>>> inputs are identical, I would expect that the output is identical as >>>> well. Is there something I am overlooking? >>>> 2. Is there a way to get consistent hash-values for functions between >>>> BATCH and interactive sessions. >>>> >>>> The output from the BATCH and interactive runs are below (as well as >>>> sessionInfo) >>>> >>>> Thank you very much for your help! >>>> >>>> Holger Hoefling >>>> >>>> --------------------------------- >>>> BATCH run (via input redirection): >>>> >>>> $ R --vanilla < testFile.R >>>> >>>> R version 3.1.0 (2014-04-10) -- "Spring Dance" >>>> Copyright (C) 2014 The R Foundation for Statistical Computing >>>> Platform: x86_64-unknown-linux-gnu (64-bit) >>>> >>>> R is free software and comes with ABSOLUTELY NO WARRANTY. >>>> You are welcome to redistribute it under certain conditions. >>>> Type 'license()' or 'licence()' for distribution details. >>>> >>>> R is a collaborative project with many contributors. >>>> Type 'contributors()' for more information and >>>> 'citation()' on how to cite R or R packages in publications. >>>> >>>> Type 'demo()' for some demos, 'help()' for on-line help, or >>>> 'help.start()' for an HTML browser interface to help. >>>> Type 'q()' to quit R. >>>> >>>>> library(digest) >>>>> testfun <- function() { >>>> >>>> >>>> + return(NULL) >>>> + } >>>>> >>>>> >>>>> print(digest(testfun)) >>>> >>>> >>>> [1] "b03160b9250f0d5b5bcce42bd86d8e56" >>>>> >>>>> >>>>> print(serialize(testfun, connection = NULL)) >>>> >>>> >>>> [1] 58 0a 00 00 00 02 00 03 01 00 00 02 03 00 00 00 04 03 00 00 00 fd 00 >>>> 00 00 >>>> [26] fe 00 00 00 06 00 00 00 01 00 04 00 09 00 00 00 01 7b 00 00 00 02 >>>> 00 >>>> 00 00 >>>> [51] 06 00 00 00 01 00 04 00 09 00 00 00 06 72 65 74 75 72 6e 00 00 00 >>>> 02 >>>> 00 00 >>>> [76] 00 fe 00 00 00 fe 00 00 00 fe >>>>> >>>>> >>>>> sessionInfo() >>>> >>>> >>>> R version 3.1.0 (2014-04-10) >>>> Platform: x86_64-unknown-linux-gnu (64-bit) >>>> >>>> locale: >>>> [1] C >>>> >>>> attached base packages: >>>> [1] stats graphics grDevices utils datasets methods base >>>> >>>> other attached packages: >>>> [1] digest_0.6.4 >>>>> >>>>> >>>>> >>>> >>>> ---------------------------------------------- >>>> Interactive run: >>>> >>>> $ R --vanilla >>>> >>>> R version 3.1.0 (2014-04-10) -- "Spring Dance" >>>> Copyright (C) 2014 The R Foundation for Statistical Computing >>>> Platform: x86_64-unknown-linux-gnu (64-bit) >>>> >>>> R is free software and comes with ABSOLUTELY NO WARRANTY. >>>> You are welcome to redistribute it under certain conditions. >>>> Type 'license()' or 'licence()' for distribution details. >>>> >>>> R is a collaborative project with many contributors. >>>> Type 'contributors()' for more information and >>>> 'citation()' on how to cite R or R packages in publications. >>>> >>>> Type 'demo()' for some demos, 'help()' for on-line help, or >>>> 'help.start()' for an HTML browser interface to help. >>>> Type 'q()' to quit R. >>>> >>>>> library(digest) >>>>> testfun <- function() { >>>> >>>> >>>> + return(NULL) >>>> + } >>>>> >>>>> >>>>> print(digest(testfun)) >>>> >>>> >>>> [1] "fada482d2894088b079a8e56b7044862" >>>>> >>>>> >>>>> print(serialize(testfun, connection = NULL)) >>>> >>>> >>>> [1] 58 0a 00 00 00 02 00 03 01 00 00 02 03 00 00 00 06 03 00 00 04 02 >>>> 00 >>>> 00 00 >>>> [26] 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 03 0d 00 00 00 >>>> 08 >>>> 00 00 >>>> [51] 00 01 00 00 00 0c 00 00 00 03 00 00 00 01 00 00 00 0c 00 00 00 01 >>>> 00 >>>> 00 00 >>>> [76] 01 00 00 00 03 00 00 04 02 00 00 00 01 00 04 00 09 00 00 00 07 73 >>>> 72 >>>> 63 66 >>>> [101] 69 6c 65 00 00 00 04 00 00 00 00 00 00 00 f2 00 00 04 02 00 00 00 >>>> 01 >>>> 00 04 >>>> [126] 00 09 00 00 00 05 6c 69 6e 65 73 00 00 00 10 00 00 00 01 00 04 00 >>>> 09 >>>> 00 00 >>>> [151] 00 2b 74 65 73 74 66 75 6e 20 3c 2d 20 66 75 6e 63 74 69 6f 6e 28 >>>> 29 >>>> 20 7b >>>> [176] 0a 20 20 20 20 72 65 74 75 72 6e 28 4e 55 4c 4c 29 0a 7d 0a 00 00 >>>> 04 >>>> 02 00 >>>> [201] 00 00 01 00 04 00 09 00 00 00 08 66 69 6c 65 6e 61 6d 65 00 00 00 >>>> 10 >>>> 00 00 >>>> [226] 00 01 00 04 00 09 00 00 00 00 00 00 00 fe 00 00 00 fe 00 00 04 02 >>>> 00 >>>> 00 00 >>>> [251] 01 00 04 00 09 00 00 00 05 63 6c 61 73 73 00 00 00 10 00 00 00 02 >>>> 00 >>>> 04 00 >>>> [276] 09 00 00 00 0b 73 72 63 66 69 6c 65 63 6f 70 79 00 04 00 09 00 00 >>>> 00 >>>> 07 73 >>>> [301] 72 63 66 69 6c 65 00 00 00 fe 00 00 04 02 00 00 06 ff 00 00 00 10 >>>> 00 >>>> 00 00 >>>> [326] 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 00 fe 00 00 00 >>>> fe >>>> 00 00 >>>> [351] 00 fd 00 00 00 fe 00 00 02 06 00 00 04 02 00 00 01 ff 00 00 00 13 >>>> 00 >>>> 00 00 >>>> [376] 02 00 00 03 0d 00 00 00 08 00 00 00 01 00 00 00 17 00 00 00 01 00 >>>> 00 >>>> 00 17 >>>> [401] 00 00 00 17 00 00 00 17 00 00 00 01 00 00 00 01 00 00 04 02 00 00 >>>> 02 >>>> ff 00 >>>> [426] 00 03 ff 00 00 04 02 00 00 06 ff 00 00 00 10 00 00 00 01 00 04 00 >>>> 09 >>>> 00 00 >>>> [451] 00 06 73 72 63 72 65 66 00 00 00 fe 00 00 03 0d 00 00 00 08 00 00 >>>> 00 >>>> 02 00 >>>> [476] 00 00 05 00 00 00 02 00 00 00 10 00 00 00 05 00 00 00 10 00 00 00 >>>> 02 >>>> 00 00 >>>> [501] 00 02 00 00 04 02 00 00 02 ff 00 00 03 ff 00 00 04 02 00 00 06 ff >>>> 00 >>>> 00 00 >>>> [526] 10 00 00 00 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 00 >>>> fe >>>> 00 00 >>>> [551] 04 02 00 00 02 ff 00 00 03 ff 00 00 04 02 00 00 00 01 00 04 00 09 >>>> 00 >>>> 00 00 >>>> [576] 0b 77 68 6f 6c 65 53 72 63 72 65 66 00 00 03 0d 00 00 00 08 00 00 >>>> 00 >>>> 01 00 >>>> [601] 00 00 00 00 00 00 03 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 >>>> 01 >>>> 00 00 >>>> [626] 00 03 00 00 04 02 00 00 02 ff 00 00 03 ff 00 00 04 02 00 00 06 ff >>>> 00 >>>> 00 00 >>>> [651] 10 00 00 00 01 00 04 00 09 00 00 00 06 73 72 63 72 65 66 00 00 00 >>>> fe >>>> 00 00 >>>> [676] 00 fe 00 00 00 01 00 04 00 09 00 00 00 01 7b 00 00 00 02 00 00 00 >>>> 06 >>>> 00 00 >>>> [701] 00 01 00 04 00 09 00 00 00 06 72 65 74 75 72 6e 00 00 00 02 00 00 >>>> 00 >>>> fe 00 >>>> [726] 00 00 fe 00 00 00 fe >>>>> >>>>> >>>>> sessionInfo() >>>> >>>> >>>> R version 3.1.0 (2014-04-10) >>>> Platform: x86_64-unknown-linux-gnu (64-bit) >>>> >>>> locale: >>>> [1] C >>>> >>>> attached base packages: >>>> [1] stats graphics grDevices utils datasets methods base >>>> >>>> other attached packages: >>>> [1] digest_0.6.4 >>>>> >>>>> >>>>> >>>> >>>> ______________________________________________ >>>> R-devel at r-project.org mailing list >>>> https://stat.ethz.ch/mailman/listinfo/r-devel >>>> >>> >>> -- >>> Luke Tierney >>> Ralph E. Wareham Professor of Mathematical Sciences >>> University of Iowa Phone: 319-335-3386 >>> Department of Statistics and Fax: 319-335-3017 >>> Actuarial Science >>> 241 Schaeffer Hall email: luke-tierney at uiowa.edu >>> Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu >> >> > > -- > Luke Tierney > Ralph E. Wareham Professor of Mathematical Sciences > University of Iowa Phone: 319-335-3386 > Department of Statistics and Fax: 319-335-3017 > Actuarial Science > 241 Schaeffer Hall email: luke-tierney at uiowa.edu > Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu
Reasonably Related Threads
- Different serialization of functions in interactive vs. batch mode
- Different serialization of functions in interactive vs. batch mode
- Different serialization of functions in interactive vs. batch mode
- Different serialization of functions in interactive vs. batch mode
- Different serialization of functions in interactive vs. batch mode