R devel - Jul 2009 - Using svSocket with data.table

Hi r-devel,

svSocket is working well.  Short demo here of using it with data.table : 
http://www.youtube.com/watch?v=rvT8XThGA8o
If you maximise and press the HD button it should be readable.  There isn't 
any audio.

However, svSocket is currently text based IPC using dump.  Is there a 
solution which will do binary and work in the same way ?  Its nice that I 
can make R become a server any time after it starts up i.e. I'm not 
restricted to just R sessions that have been started by some cluster 
wrapper.  Its also great that the command line interface is not blocked.   I 
can even type options(debug.Socket=TRUE) into the server to see the traffic, 
then type in to set debug back to FALSE again,  all while its running 
(although that isn't in this demo).  And of course this is cross platform, 
the R client can be 32 bit windows, the R server can be 64 bit lunix.  Or 
remote graphics for example.

So I'm looking to do the same as the demo,  but with a binary socket.  Does 
anyone have any ideas?  I've looked a bit at  Rserve, bigmemory, biocep, nws
but although all those packages are great,  I didn't find anything that 
worked in exactly this way i.e.  i) R to R ii) CLI non-blocking and iii) no 
need to startup R in a special way

Regards, Matthew

Hi,

Just look at the way Rserve wraps data and the way the Rserve client 
unwraps it, and pipe the data through the svSocket instead of Rserve 
communication protocols.

The command line is not blocked because svSocket uses tcl/tk event loop 
for the communication.

So there are these questions:
- can the socket implemented in svSocket handle binary transfer of data.
- can you wrap the data suitable for transfer through a socket
- can you then unwrap the data on the other side of the socket

I am pretty sure the two last questions can be answered by Rserve. The 
first question, you can answer by looking at the way the socket server 
is created, and then change it so it transfers binary data, I would be 
surprised if the tcl socket would not be able to transfer binary data.

The "danger" of svSocket is that it somehow gives the illusion that R 
deals with concurrent requests (from the command line and the socket)

Romain


On 07/25/2009 09:07 AM, Matthew Dowle wrote:
> Hi r-devel,
>
> svSocket is working well.  Short demo here of using it with data.table :
> http://www.youtube.com/watch?v=rvT8XThGA8o
> If you maximise and press the HD button it should be readable.  There
isn't
> any audio.
>
> However, svSocket is currently text based IPC using dump.  Is there a
> solution which will do binary and work in the same way ?  Its nice that I
> can make R become a server any time after it starts up i.e. I'm not
> restricted to just R sessions that have been started by some cluster
> wrapper.  Its also great that the command line interface is not blocked.  
I
> can even type options(debug.Socket=TRUE) into the server to see the
traffic,
> then type in to set debug back to FALSE again,  all while its running
> (although that isn't in this demo).  And of course this is cross
platform,
> the R client can be 32 bit windows, the R server can be 64 bit lunix.  Or
> remote graphics for example.
>
> So I'm looking to do the same as the demo,  but with a binary socket. 
Does
> anyone have any ideas?  I've looked a bit at  Rserve, bigmemory,
biocep, nws
> but although all those packages are great,  I didn't find anything that
> worked in exactly this way i.e.  i) R to R ii) CLI non-blocking and iii) no
> need to startup R in a special way
>
> Regards, Matthew

-- 
Romain Francois
Independent R Consultant
+33(0) 6 28 91 30 30
http://romainfrancois.blog.free.fr
|- http://tr.im/tlNb : RGG#155, 156 and 157
|- http://tr.im/rw0p : useR! slides
`- http://tr.im/rw0b : RGG#154: demo of atomic functions

Hi Matthew,

Excerpts from Matthew Dowle's message of Sat Jul 25 09:07:44 +0200
2009:
> So I'm looking to do the same as the demo,  but with a binary socket. 
Does
> anyone have any ideas?  I've looked a bit at  Rserve, bigmemory,
biocep, nws
> but although all those packages are great,  I didn't find anything that
> worked in exactly this way i.e.  i) R to R ii) CLI non-blocking and iii) no
> need to startup R in a special way
Don't be fooled. R does not handle multiple requests in parallel
internally. Also I suspect that, depending on what you do on the CLI,
this will interact badly with svSocket. 

As far as binary transfer of R objects goes, you are probably looking
for serialize() and unserialize(). Not sure if these are guaranteed to
work across differen versions of R and different word sizes. See the
Warnings section in the serialize manual page.

Cheers
Olaf