On Mon, 25 Jun 2007, Ei-ji Nakama wrote:
> problem of the very large memory require by the Sign extension.
>
> --- R-2.5.0.orig/src/main/serialize.c 2007-03-27 01:42:08.000000000 +0900
> +++ R-2.5.0/src/main/serialize.c 2007-06-25 00:48:58.000000000 +0900
> @@ -1866,7 +1866,7 @@
>
> static void resize_buffer(membuf_t mb, int needed)
> {
> - int newsize = 2 * needed;
> + size_t newsize = 2 * needed;
> mb->buf = realloc(mb->buf, newsize);
> if (mb->buf == NULL)
> error(_("cannot allocate buffer"));
Yes, thanks, but the structure also needs to be changed as the next line
is
mb->size = newsize;
and so this would set mb->size to a negative value.
Could you please tell us where you encountered this?
As far as I can see, this code is only used via R_serialize for
serializing to a raw vector, in which case 'size' can safely be
'int' and
the re-allocation should be up to 2^31-1 bytes at most (and allocating
twice what you are asked for seems undesirable). But there is potential
overflow at
if (mb->count + length > mb->size)
and possibly elsewhere.
> The time-out of read and write was not set.
>
> 51:sendData.SOCKnode <- function(node, data) {
> 52: timeout <- getClusterOption("timeout")
> 53: old <- options(timeout = timeout);
> 54: on.exit(options(old))
> 55: serialize(data, node$con)
> 56: }
> 57:
> 58:recvData.SOCKnode <- function(node) {
> 59: timeout <- getClusterOption("timeout")
> 60: old <- options(timeout = timeout);
> 61: on.exit(options(old))
> 62: unserialize(node$con)
> 63: }
I don't think sock_read/sock_write is the right place to make that
setting. Ideally it would be set when the option is set, but as this is
in a module that needs an extension to the interface.
Looking at the code, we read from a socket in blocks of 4096, but we write
in a single block. The latter is likely to be the problem, and I think
some redesigning is necessary here.
Perhaps you and Luke Tierney can comment on exactly what the problem is
and how best to work around it.
>
> --- R-2.5.0.orig/src/modules/internet/sockconn.c 2006-09-04
23:20:59.000000000 +0900
> +++ R-2.5.0/src/modules/internet/sockconn.c 2007-06-25
> 00:51:38.000000000 +0900
> @@ -155,14 +155,19 @@
> static size_t sock_read(void *ptr, size_t size, size_t nitems,
> Rconnection con)
> {
> + int timeout = asInteger(GetOption(install("timeout"),
R_BaseEnv));
> +
> + R_SockTimeout(timeout);
> return sock_read_helper(con, ptr, size * nitems)/size;
> }
>
> static size_t sock_write(const void *ptr, size_t size, size_t nitems,
> Rconnection con)
> {
> + int timeout = asInteger(GetOption(install("timeout"),
R_BaseEnv));
> Rsockconn this = (Rsockconn)con->private;
>
> + R_SockTimeout(timeout);
> return R_SockWrite(this->fd, ptr, size * nitems)/size;
> }
>
>
--
Brian D. Ripley, ripley at stats.ox.ac.uk
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UK Fax: +44 1865 272595