Displaying 5 results from an estimated 5 matches for "initmemoutpstream".
2015 Mar 17
2
Reduce memory peak when serializing to raw vectors
...peak during serialize() is ~1.2GB:
$ cat /proc/15155/status |grep Vm
...
VmHWM: 1207792 kB
VmRSS: 817272 kB
We work with very large data frames and in many cases this is killing R with an "out of memory" error.
This is the relevant code in R 3.1.3 in src/main/serialize.c:2494
InitMemOutPStream(&out, &mbs, type, version, hook, fun);
R_Serialize(object, &out);
val = CloseMemOutPStream(&out);
The serialized object is being stored in a buffer pointed by out.data. Then in CloseMemOutPStream() R copies the whole buffer to a newly allocated SEXP object (the raw vector that s...
2015 Mar 17
2
Reduce memory peak when serializing to raw vectors
...1207792 kB
> > VmRSS: 817272 kB
> >
> > We work with very large data frames and in many cases this is killing R
> with an "out of memory" error.
> >
> > This is the relevant code in R 3.1.3 in src/main/serialize.c:2494
> >
> > InitMemOutPStream(&out, &mbs, type, version, hook, fun);
> > R_Serialize(object, &out);
> > val = CloseMemOutPStream(&out);
> >
> > The serialized object is being stored in a buffer pointed by out.data.
> Then in CloseMemOutPStream() R copies the whole buffer t...
2015 Mar 17
0
Reduce memory peak when serializing to raw vectors
...proc/15155/status |grep Vm
> ...
> VmHWM: 1207792 kB
> VmRSS: 817272 kB
>
> We work with very large data frames and in many cases this is killing R with an "out of memory" error.
>
> This is the relevant code in R 3.1.3 in src/main/serialize.c:2494
>
> InitMemOutPStream(&out, &mbs, type, version, hook, fun);
> R_Serialize(object, &out);
> val = CloseMemOutPStream(&out);
>
> The serialized object is being stored in a buffer pointed by out.data. Then in CloseMemOutPStream() R copies the whole buffer to a newly allocated SEXP object (t...
2015 Mar 17
0
Reduce memory peak when serializing to raw vectors
...peak during serialize() is ~1.2GB:
$ cat /proc/15155/status |grep Vm
...
VmHWM: 1207792 kB
VmRSS: 817272 kB
We work with very large data frames and in many cases this is killing R with an "out of memory" error.
This is the relevant code in R 3.1.3 in src/main/serialize.c:2494
InitMemOutPStream(&out, &mbs, type, version, hook, fun);
R_Serialize(object, &out);
val = CloseMemOutPStream(&out);
The serialized object is being stored in a buffer pointed by out.data. Then in CloseMemOutPStream() R copies the whole buffer to a newly allocated SEXP object (the raw vector that s...
2015 Mar 17
0
Reduce memory peak when serializing to raw vectors
...> VmRSS: 817272 kB
>> >
>> > We work with very large data frames and in many cases this is killing R with an "out of memory" error.
>> >
>> > This is the relevant code in R 3.1.3 in src/main/serialize.c:2494
>> >
>> > InitMemOutPStream(&out, &mbs, type, version, hook, fun);
>> > R_Serialize(object, &out);
>> > val = CloseMemOutPStream(&out);
>> >
>> > The serialized object is being stored in a buffer pointed by out.data. Then in CloseMemOutPStream() R copies the who...