Philippe Grosjean
2009-Aug-27 07:09 UTC
[R] [Fwd: Re: Video demo of using svSocket with data.table]
Forwarded to R-Help, because I think it could interest people following this thread. Clearly, RServe and svSocket have different goals and very little overlap. Best, Philippe -------- Original Message -------- Subject: Re: Video demo of using svSocket with data.table Date: Wed, 26 Aug 2009 20:34:19 +0100 From: Matthew Dowle <mdowle at mdowle.plus.com> Reply-To: Matthew Dowle <mdowle at mdowle.plus.com> To: Philippe Grosjean <phgrosjean at sciviews.org>, Romain Francois <romain.francois at dbmail.com> References: <h6kcod$5ed$1 at ger.gmane.org> <4A8E632D.6060504 at sciviews.org><4A8E6AF1.9060009 at dbmail.com> <4A8E9E7D.9070209 at sciviews.org> Hi Philippe & Romain, Thanks - interesting discussion on the list - I just caught up with it now. I agree with everything basically and look forward to the 'planned for the future'. The 'feature' of "every clients and the CLI are working with the same objects on the same environment" is really important to keep btw - got a bit worried by your "*feature*" in case you thought it was a bad thing. Rserve doesn't do that! Its a great feature and really important. I compared Rserve to svSocket and came up with this list (confirmed with Simon U). You probably already know this but just in case : 1. The main thing is Rserve's clients each have their own workspace. Its not one shared workspace, unlike with svSocket. So one client can't write something and another client then read it because they only see their own workspaces within Rserve. You might as well start lots of R's basically. 2. There is no CLI (command line interface) to Rserve i.e. no prompt to type at. Its just a process that sits there and responds to clients only. 3. Rserve on windows is limited to only one client connection, no more. The docs say that Windows is not recommended (for this reason). 4. You have to start Rserve first before you send commands to it. With svSocket, you can startup any old R, do some analysis and gather data, then decide to become a server and let clients connect. This is a really important workflow feature. With Rserve you have to think in advance and know that you'll need to be a server. You can't do any of those things with Rserve, but you can with svSocket. Rserve does do binary data transfer though. Regards, Matthew ----- Original Message ----- From: "Philippe Grosjean" <phgrosjean at sciviews.org> Newsgroups: gmane.comp.lang.r.general To: "Romain Francois" <romain.francois at dbmail.com> Cc: "Matthew Dowle" <mdowle at mdowle.plus.com>; <r-help at stat.math.ethz.ch> Sent: Friday, August 21, 2009 2:17 PM Subject: Re: Video demo of using svSocket with data.table Romain Francois wrote:> Hi Philippe, > > When Matthew brought this up the first time on this list, there were > several replies to warn about potential problems related to R not being > thread safe, and that this might cause trouble.Well, that is true, R is not thread safe. What happens, basically, is that R clients run in the tcltk event loop. When a client is doing something, R is locked, processing the client's request before returning to the main loop. On the contrary, something running in the main loop can be interrupted pretty much anytime by a client's request. Regarding different clients, it is first in, first served rule: requests are processed in the order they appear. It would be possible to adapt svSocket to delay processing of (asynchronous only) requests from clients, waiting from flags set by, either the main loop, or another client. That would be rather easy to do. Otherwise, every clients and the CLI are working with the same objects on the same environment (.GlobalEnv, as primary one), but that is a *feature*! One constraint for designing svSocket is that the behaviour of R has to be as much as possible identical when a command is run on the main loop through the CLI, or from within a client. Of course, you can imagine all sorts of bad interactions, and it is very, very easy to write a bad-behaving client. The goal is not to write a client-server architecture, but a multitasking way of manipulating R by the same end-user (thus, not likely to feed bad code from one side to destroy what he is doing from another side :-) Best, Philippe> Since you were on holidays, we did not get your viewpoint. Could you > elaborate on how you deal with this. > > "browser" works off the REPL, so this is unlikely that svSocket can take > advantage of it, since the socket runs on a different loop. or maybe you > can add something that feeds the R main loop, but I'm not sure this is > possible unless you embed R ... > > Romain > > On 08/21/2009 11:04 AM, Philippe Grosjean wrote: >> >> Hello Matthew and all R-UseRs, >> >> You video demo is very nice. This suggests various uses of svSocket that >> I had not think about! The primary goal was to make it: >> - flexible (I think it is clear from the demo), >> - running in the background while not blocking the CLI (Rgui, R.app, or >> the terminal, very clear from your demo too), >> - stateful (yes, this is not in your demo, but a client can disconnect >> and reconnect and got the same server state it had just before >> disconnection, including possibly, partial command send to R server), >> >> Not implemented yet, but planned for the future: >> - binary transfer of R objects, >> - connection to distant secured server using TSL (of course, distant >> connection requires a lot of extra precautions because R is NOT an >> Internet-secure language and environment, but that applies to all >> client/server R solutions like Rserve or Rpad), >> - mirroring of the commands, results and history on the different >> clients to make a simple collaborative R session. >> >> The primary use in SciViews is the communication engine between the >> client (a code editor, or IDE program like Komodo Edit) and server (R). >> Your demo gives an idea on the flexibility one got with it, including >> the possibility to inspect and/or change objects while R is running a >> long process. Your example of changing the plot in real-time without >> interrupting the main server's process is very illustrative. So now, >> imagine the debugging flexibility for long running tasks, and/or >> combination of svSocket with browser()... But that's another story, >> because svSocket does not work nicely with browser() for the moment. >> >> All the best, >> >> Philippe >> >> ..............................................<?}))><........ >> ) ) ) ) ) >> ( ( ( ( ( Prof. Philippe Grosjean >> ) ) ) ) ) >> ( ( ( ( ( Numerical Ecology of Aquatic Systems >> ) ) ) ) ) Mons-Hainaut University, Belgium >> ( ( ( ( ( >> .............................................................. >> >> Matthew Dowle wrote: >>> Dear r-help, >>> If you haven't already seen this then : >>> http://www.youtube.com/watch?v=rvT8XThGA8o >>> The video consists of typing at the console and graphics, there is no >>> audio >>> or slides. Please press the HD button and maximise. Its about 8 mins. >>> Regards, Matthew >-- ..............................................<?}))><........ ) ) ) ) ) ( ( ( ( ( Prof. Philippe Grosjean ) ) ) ) ) ( ( ( ( ( Numerical Ecology of Aquatic Systems ) ) ) ) ) Mons-Hainaut University, Belgium ( ( ( ( ( ..............................................................
Erich Neuwirth
2009-Aug-27 10:11 UTC
[R] [Fwd: Re: Video demo of using svSocket with data.table]
If you are working on windows, Thomas Baier's statconnDCOM and rcom allow you to access R servers via COM. The statconnDCOM package also has a servermanager which allows to configure servers for exclusive or nonexclusive (= common workspace) usage for different clients (possibly living on different machines). Philippe Grosjean wrote:> Forwarded to R-Help, because I think it could interest people following > this thread. Clearly, RServe and svSocket have different goals and very > little overlap. > > Best, > > Philippe > > > > -------- Original Message -------- > Subject: Re: Video demo of using svSocket with data.table > Date: Wed, 26 Aug 2009 20:34:19 +0100 > From: Matthew Dowle <mdowle at mdowle.plus.com> > Reply-To: Matthew Dowle <mdowle at mdowle.plus.com> > To: Philippe Grosjean <phgrosjean at sciviews.org>, Romain Francois > <romain.francois at dbmail.com> > References: <h6kcod$5ed$1 at ger.gmane.org> > <4A8E632D.6060504 at sciviews.org><4A8E6AF1.9060009 at dbmail.com> > <4A8E9E7D.9070209 at sciviews.org> > > Hi Philippe & Romain, > > Thanks - interesting discussion on the list - I just caught up with it now. > I agree with everything basically and look forward to the 'planned for the > future'. The 'feature' of "every clients and the CLI are working with the > same objects on the same environment" is really important to keep btw - got > a bit worried by your "*feature*" in case you thought it was a bad thing. > Rserve doesn't do that! Its a great feature and really important. > > I compared Rserve to svSocket and came up with this list (confirmed with > Simon U). You probably already know this but just in case : > > 1. The main thing is Rserve's clients each have their own workspace. Its > not one shared workspace, unlike with svSocket. So one client can't write > something and another client then read it because they only see their own > workspaces within Rserve. You might as well start lots of R's basically. > 2. There is no CLI (command line interface) to Rserve i.e. no prompt to > type > at. Its just a process that sits there and responds to clients only. > 3. Rserve on windows is limited to only one client connection, no more. The > docs say that Windows is not recommended (for this reason). > 4. You have to start Rserve first before you send commands to it. With > svSocket, you can startup any old R, do some analysis and gather data, then > decide to become a server and let clients connect. This is a really > important workflow feature. With Rserve you have to think in advance and > know that you'll need to be a server. > > You can't do any of those things with Rserve, but you can with svSocket. > Rserve does do binary data transfer though. > > Regards, Matthew > > ----- Original Message ----- > From: "Philippe Grosjean" <phgrosjean at sciviews.org> > Newsgroups: gmane.comp.lang.r.general > To: "Romain Francois" <romain.francois at dbmail.com> > Cc: "Matthew Dowle" <mdowle at mdowle.plus.com>; <r-help at stat.math.ethz.ch> > Sent: Friday, August 21, 2009 2:17 PM > Subject: Re: Video demo of using svSocket with data.table > > > Romain Francois wrote: >> Hi Philippe, >> >> When Matthew brought this up the first time on this list, there were >> several replies to warn about potential problems related to R not >> being thread safe, and that this might cause trouble. > > Well, that is true, R is not thread safe. What happens, basically, is > that R clients run in the tcltk event loop. When a client is doing > something, R is locked, processing the client's request before returning > to the main loop. On the contrary, something running in the main loop > can be interrupted pretty much anytime by a client's request. Regarding > different clients, it is first in, first served rule: requests are > processed in the order they appear. > > It would be possible to adapt svSocket to delay processing of > (asynchronous only) requests from clients, waiting from flags set by, > either the main loop, or another client. That would be rather easy to do. > > Otherwise, every clients and the CLI are working with the same objects > on the same environment (.GlobalEnv, as primary one), but that is a > *feature*! One constraint for designing svSocket is that the behaviour > of R has to be as much as possible identical when a command is run on > the main loop through the CLI, or from within a client. > > Of course, you can imagine all sorts of bad interactions, and it is > very, very easy to write a bad-behaving client. The goal is not to write > a client-server architecture, but a multitasking way of manipulating R > by the same end-user (thus, not likely to feed bad code from one side to > destroy what he is doing from another side :-) > Best, > > Philippe > >> Since you were on holidays, we did not get your viewpoint. Could you >> elaborate on how you deal with this. >> >> "browser" works off the REPL, so this is unlikely that svSocket can >> take advantage of it, since the socket runs on a different loop. or >> maybe you can add something that feeds the R main loop, but I'm not >> sure this is possible unless you embed R ... >> >> Romain >> >> On 08/21/2009 11:04 AM, Philippe Grosjean wrote: >>> >>> Hello Matthew and all R-UseRs, >>> >>> You video demo is very nice. This suggests various uses of svSocket that >>> I had not think about! The primary goal was to make it: >>> - flexible (I think it is clear from the demo), >>> - running in the background while not blocking the CLI (Rgui, R.app, or >>> the terminal, very clear from your demo too), >>> - stateful (yes, this is not in your demo, but a client can disconnect >>> and reconnect and got the same server state it had just before >>> disconnection, including possibly, partial command send to R server), >>> >>> Not implemented yet, but planned for the future: >>> - binary transfer of R objects, >>> - connection to distant secured server using TSL (of course, distant >>> connection requires a lot of extra precautions because R is NOT an >>> Internet-secure language and environment, but that applies to all >>> client/server R solutions like Rserve or Rpad), >>> - mirroring of the commands, results and history on the different >>> clients to make a simple collaborative R session. >>> >>> The primary use in SciViews is the communication engine between the >>> client (a code editor, or IDE program like Komodo Edit) and server (R). >>> Your demo gives an idea on the flexibility one got with it, including >>> the possibility to inspect and/or change objects while R is running a >>> long process. Your example of changing the plot in real-time without >>> interrupting the main server's process is very illustrative. So now, >>> imagine the debugging flexibility for long running tasks, and/or >>> combination of svSocket with browser()... But that's another story, >>> because svSocket does not work nicely with browser() for the moment. >>> >>> All the best, >>> >>> Philippe >>> >>> ..............................................<?}))><........ >>> ) ) ) ) ) >>> ( ( ( ( ( Prof. Philippe Grosjean >>> ) ) ) ) ) >>> ( ( ( ( ( Numerical Ecology of Aquatic Systems >>> ) ) ) ) ) Mons-Hainaut University, Belgium >>> ( ( ( ( ( >>> .............................................................. >>> >>> Matthew Dowle wrote: >>>> Dear r-help, >>>> If you haven't already seen this then : >>>> http://www.youtube.com/watch?v=rvT8XThGA8o >>>> The video consists of typing at the console and graphics, there is no >>>> audio >>>> or slides. Please press the HD button and maximise. Its about 8 mins. >>>> Regards, Matthew >> > > > > >-- Erich Neuwirth, University of Vienna Faculty of Computer Science Computer Supported Didactics Working Group Visit our SunSITE at http://sunsite.univie.ac.at Phone: +43-1-4277-39464 Fax: +43-1-4277-39459