Displaying 10 results from an estimated 10 matches for "r_serial".
Did you mean:
u_serial
2009 Mar 31
1
external equiv to R_serialize()?
I'm trying to efficiently allow conversion of R objects to PostgreSQL
bytea (raw binary) datatype within PL/R for persistent storage in
Postgres tables. I have found R_serialize() which looks like what I
need, -- e.g. R_serialize(object, NULL, FALSE, NULL) -- except that it
is marked attribute_hidden. Is there some other externally available
interface that I can use?
Thanks,
Joe
2012 Mar 22
1
Serializing many small objects efficiently
...asonably experienced in each separately).
I am trying to write a serialization function for a format called
typedbytes, which is used as an interchange format in Hadoop circles. Since
I would need to serialize according to the internal R format many small R
objects I looked at the c interface
void R_Serialize(SEXP s, R_outpstream_t ops);
SEXP R_Unserialize(R_inpstream_t ips);
If I look at the source for e.g. unserialize is see a
.Call("R_unserialize", connection, refhook, PACKAGE = "base")
which, despite the name of the second argument, accepts as 'connection' a
raw ve...
2015 Mar 17
2
Reduce memory peak when serializing to raw vectors
...ep Vm
...
VmHWM: 1207792 kB
VmRSS: 817272 kB
We work with very large data frames and in many cases this is killing R with an "out of memory" error.
This is the relevant code in R 3.1.3 in src/main/serialize.c:2494
InitMemOutPStream(&out, &mbs, type, version, hook, fun);
R_Serialize(object, &out);
val = CloseMemOutPStream(&out);
The serialized object is being stored in a buffer pointed by out.data. Then in CloseMemOutPStream() R copies the whole buffer to a newly allocated SEXP object (the raw vector that stores the final result):
PROTECT(val = allocVector(RAWS...
2006 Feb 23
1
Utilizing the internet module
Hello all,
I'd like to utilize the R_Sock* functions from R_ext/R-ftp-http.h in my
R package. The intent is to use these in conjunction with R_serialize()
to store R objects in a remote data store. I'm aware that version
2.2.1 of "Writing R extensions" explains that these may be undocumented
and unstable, but I have a couple of questions:
1) are they platform independent? I presume they are...
2) What's the appropriate w...
2015 Mar 17
2
Reduce memory peak when serializing to raw vectors
...h very large data frames and in many cases this is killing R
> with an "out of memory" error.
> >
> > This is the relevant code in R 3.1.3 in src/main/serialize.c:2494
> >
> > InitMemOutPStream(&out, &mbs, type, version, hook, fun);
> > R_Serialize(object, &out);
> > val = CloseMemOutPStream(&out);
> >
> > The serialized object is being stored in a buffer pointed by out.data.
> Then in CloseMemOutPStream() R copies the whole buffer to a newly allocated
> SEXP object (the raw vector that stores the fin...
2007 Nov 02
0
applying duplicated, unique and match to lists?
...d.array, match.array (pairs of projections may erroneously apear as identical when the vectors are very similar, RAM expensive)
pasteid <- function(x)paste(x, collapse="\r")
# perfectly identity preserving projection (but even more RAM expensive)
id1 <- function(x)paste(.Call("R_serialize", x, NULL, FALSE, NULL, PACKAGE = "base")[-(1:14)], collapse="")
# 32 byte projection
md5 <- function(x).Call("digest", .Call("R_serialize", x, NULL, FALSE, NULL, PACKAGE = "base"), 1L, -1L, 14L, PACKAGE = "digest")
# 40 byte...
2015 Mar 17
0
Reduce memory peak when serializing to raw vectors
...SS: 817272 kB
>
> We work with very large data frames and in many cases this is killing R with an "out of memory" error.
>
> This is the relevant code in R 3.1.3 in src/main/serialize.c:2494
>
> InitMemOutPStream(&out, &mbs, type, version, hook, fun);
> R_Serialize(object, &out);
> val = CloseMemOutPStream(&out);
>
> The serialized object is being stored in a buffer pointed by out.data. Then in CloseMemOutPStream() R copies the whole buffer to a newly allocated SEXP object (the raw vector that stores the final result):
>
> PROTE...
2015 Mar 17
0
Reduce memory peak when serializing to raw vectors
...ep Vm
...
VmHWM: 1207792 kB
VmRSS: 817272 kB
We work with very large data frames and in many cases this is killing R with an "out of memory" error.
This is the relevant code in R 3.1.3 in src/main/serialize.c:2494
InitMemOutPStream(&out, &mbs, type, version, hook, fun);
R_Serialize(object, &out);
val = CloseMemOutPStream(&out);
The serialized object is being stored in a buffer pointed by out.data. Then in CloseMemOutPStream() R copies the whole buffer to a newly allocated SEXP object (the raw vector that stores the final result):
PROTECT(val = allocVector(RAWS...
2015 Mar 17
0
Reduce memory peak when serializing to raw vectors
...ta frames and in many cases this is killing R with an "out of memory" error.
>> >
>> > This is the relevant code in R 3.1.3 in src/main/serialize.c:2494
>> >
>> > InitMemOutPStream(&out, &mbs, type, version, hook, fun);
>> > R_Serialize(object, &out);
>> > val = CloseMemOutPStream(&out);
>> >
>> > The serialized object is being stored in a buffer pointed by out.data. Then in CloseMemOutPStream() R copies the whole buffer to a newly allocated SEXP object (the raw vector that stores the f...
2007 Jun 24
1
There was a problem by the use of snow.
problem of the very large memory require by the Sign extension.
--- R-2.5.0.orig/src/main/serialize.c 2007-03-27 01:42:08.000000000 +0900
+++ R-2.5.0/src/main/serialize.c 2007-06-25 00:48:58.000000000 +0900
@@ -1866,7 +1866,7 @@
static void resize_buffer(membuf_t mb, int needed)
{
- int newsize = 2 * needed;
+ size_t newsize = 2 * needed;
mb->buf = realloc(mb->buf,