Hi everyone,
I have 2 environments (2 different R sessions) as described below:
Session 1:
Name of the environment: "CrlmmInfo"
Objects in the environment:
index1: logical index - length 238304
index2: logical index - length 238304
priors: list of 4 - (matrix 6x6, 2 vectors of length 6, vector of
length 2) - all num
params: list of 4:
centers [238304 x 3 x 2]: num
scales [238304 x 3 x 2]: num
N [238304 x 3]: num
f0 [scalar]: num
If I save this environment to a file, I get a file of 23MB. Great.
Session 2:
Analogous to "Session 1", but replace 238304 by 262264.
If I save the environment on Session 2, I get a file of 8.4GB.
I applied object.size on each of the objects in each environment, and
this is what I got:
For Session 1:
index1: 16204864
index2: 16204864
priors: 3336
params: 74353584
For Session 2:
index1: 1049096
index2: 1049096
priors: 3336
params: 81829104
Is this increase from 23MB to 8.4GB expected to happen?
Benilton
sessionInfo() on both sessions is identical:
> sessionInfo()
R version 2.4.0 (2006-10-03)
x86_64-unknown-linux-gnu
locale:
LC_CTYPE=en_US.iso885915;LC_NUMERIC=C;LC_TIME=en_US.iso885915;LC_COLLATE
=en_US.iso885915;LC_MONETARY=en_US.iso885915;LC_MESSAGES=en_US.iso885915
;LC_PAPER=en_US.iso885915;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASU
REMENT=en_US.iso885915;LC_IDENTIFICATION=C
attached base packages:
[1] "methods" "stats" "graphics"
"grDevices" "utils"
"datasets"
[7] "base"
> version
_
platform x86_64-unknown-linux-gnu
arch x86_64
os linux-gnu
system x86_64, linux-gnu
status
major 2
minor 4.0
year 2006
month 10
day 03
svn rev 39566
language R
version.string R version 2.4.0 (2006-10-03)
[Moved to r-devel for further discussion, bcc:ed r-help for the record]
Hi Benilton,
it is possible that this has to do with so called unevaluated
promises. I had a similar problem a few months ago. I emailed R-devel
about it - "[Rd] save() saves extra stuff if object is not evaluated"
(Thu May 25 09:19:28 CEST 2006) [
https://stat.ethz.ch/pipermail/r-devel/2006-May/037859.html ], and
Luke Tierney gave a very good explanation on what was going on. Since
then save() has gained an extra argument 'eval.promises=TRUE', cf. R
v2.4.0 NEWS [ http://cran.at.r-project.org/ ]:
o save() by default evaluates promise objects. The old behaviour
(to save the promise and its evaluation environment) can be
obtained by setting the new argument 'eval.promises' to FALSE.
(Note that this does not apply to promises embedded in
objects, only to top-level objects.)
You are already running R v2.4.0 so this should apply to your session
already. However, note the last sentense in parentheses; it does not
handle promises in embedded objects. You could try to evaluate your
environments/promises manually using, say, is.null(env) (as I did in
my example in my May message) and see if it makes a difference.
Hope this helps
Henrik
On 11/14/06, Benilton Carvalho <bcarvalh at jhsph.edu>
wrote:> Hi everyone,
>
> I have 2 environments (2 different R sessions) as described below:
>
> Session 1:
>
> Name of the environment: "CrlmmInfo"
> Objects in the environment:
> index1: logical index - length 238304
> index2: logical index - length 238304
> priors: list of 4 - (matrix 6x6, 2 vectors of length 6, vector of
> length 2) - all num
> params: list of 4:
> centers [238304 x 3 x 2]: num
> scales [238304 x 3 x 2]: num
> N [238304 x 3]: num
> f0 [scalar]: num
>
> If I save this environment to a file, I get a file of 23MB. Great.
>
> Session 2:
> Analogous to "Session 1", but replace 238304 by 262264.
>
> If I save the environment on Session 2, I get a file of 8.4GB.
>
> I applied object.size on each of the objects in each environment, and
> this is what I got:
>
> For Session 1:
> index1: 16204864
> index2: 16204864
> priors: 3336
> params: 74353584
>
> For Session 2:
> index1: 1049096
> index2: 1049096
> priors: 3336
> params: 81829104
>
> Is this increase from 23MB to 8.4GB expected to happen?
>
> Benilton
>
> sessionInfo() on both sessions is identical:
>
> > sessionInfo()
> R version 2.4.0 (2006-10-03)
> x86_64-unknown-linux-gnu
>
> locale:
> LC_CTYPE=en_US.iso885915;LC_NUMERIC=C;LC_TIME=en_US.iso885915;LC_COLLATE
> =en_US.iso885915;LC_MONETARY=en_US.iso885915;LC_MESSAGES=en_US.iso885915
> ;LC_PAPER=en_US.iso885915;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASU
> REMENT=en_US.iso885915;LC_IDENTIFICATION=C
>
> attached base packages:
> [1] "methods" "stats" "graphics"
"grDevices" "utils"
> "datasets"
> [7] "base"
>
> > version
> _
> platform x86_64-unknown-linux-gnu
> arch x86_64
> os linux-gnu
> system x86_64, linux-gnu
> status
> major 2
> minor 4.0
> year 2006
> month 10
> day 03
> svn rev 39566
> language R
> version.string R version 2.4.0 (2006-10-03)
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
On Mon, 13 Nov 2006, Benilton Carvalho wrote:> Hi everyone, > > I have 2 environments (2 different R sessions) as described below: > > Session 1: > > Name of the environment: "CrlmmInfo" > Objects in the environment: > index1: logical index - length 238304 > index2: logical index - length 238304 > priors: list of 4 - (matrix 6x6, 2 vectors of length 6, vector of > length 2) - all num > params: list of 4: > centers [238304 x 3 x 2]: num > scales [238304 x 3 x 2]: num > N [238304 x 3]: num > f0 [scalar]: num > > If I save this environment to a file, I get a file of 23MB. Great. > > Session 2: > Analogous to "Session 1", but replace 238304 by 262264. > > If I save the environment on Session 2, I get a file of 8.4GB. > > I applied object.size on each of the objects in each environment, and > this is what I got: > > For Session 1: > index1: 16204864 > index2: 16204864 > priors: 3336 > params: 74353584 > > For Session 2: > index1: 1049096 > index2: 1049096 > priors: 3336 > params: 81829104 > > Is this increase from 23MB to 8.4GB expected to happen?We don't have enough information to expect anything. Saving environments is a tricky business: read the description in the R Internals manual if this is news to you. But note that the index[12] objects are much smaller in Session 2, and those in Session 1 are about 16x larger than needed for the description given. So my guess is that they have attributes which are promises in Session 2, and save() there is pulling in another environment to enable those attributes to be evaluated later. -- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595