Jeff Ryan
2009-May-18 17:40 UTC
[Rd] readBin on binary non-blocking connections (Windows & Unix differences/bugs)
R-devel: I am encountering a consistency issue using socketConnection and readBin with *non-blocking* connections on Unix and Windows XP (no Vista to test). I am a bit confused by the behavior of *non-blocking* connections under Windows specifically. When calling readBin on a non-blocking connection when there is no data to read on the socket, the connection under Unix will return a vector of zero-length matching the type of the 'what' argument to the readBin call. The same call under Windows returns random data unless called with what=raw(), which instead returns an error. A simple example using two R processes to illustrate on one machine: ######################################################################### # PROCESS 1 (Windows/Unix -- the same code for both): ######################################################################### # Open the connection and don't write anything s <- socketConnection(port=7777, server=TRUE, blocking=TRUE,open="ab") ######################################################################### # PROCESS 2 (Windows) use this to read: ######################################################################### s <- socketConnection(port=7777,blocking=FALSE,open="ab") readBin(s, character()) readBin(s, character()) readBin(s, double()) readBin(s, raw()) # > readBin(s, character()) # [1] "\020??\001@??\001 $?\001 $?\001 $?\001 $?\001 $?\001 $?\001 $?\001 $?\001 ... # > readBin(s, character()) # [1] "X\n" # > readBin(s, double()) # [1] 1.255609e-316 # > readBin(s, raw()) # Error in readBin(s, raw()) : negative length vectors are not allowed ########################################################################## # Using a *nix, the above works as I would expect # # PROCESS 2 (Unixes -- this example OSX 10.4, but is consistent on other flavors) ######################################################################### s <- socketConnection(port=7777,blocking=FALSE, open="ab") readBin(s, character()) readBin(s, raw()) readBin(s, double()) # > readBin(s, character()) # character(0) # > readBin(s, raw()) # raw(0) # > readBin(s, double()) # numeric(0) Is this a bug in R or Windows? Is there a workaround? I would like to have readBin on non-blocking connections work consistently across platforms. When data is present, both platforms behave consistently. Incidentally, the readBin calls in my program are not using R as the server, the above code is simply to illustrate the problem. The fact that the raw() returns an error on Windows seems to indicate that the error is able to be caught, and could be tested for. One other interesting point of note; the character() calls on Windows will return not just random data, but very non-random R type strings. Somehow non-allocated memory is being read.> readBin(s, character(),100)[1] "X\n" [2] "spatch" [3] "" [4] "" [5] "X\n" [6] "ssion" [7] "\002" [8] "?$?\001?|?\001\030[?\001?N?\001?N?\001?$?\001\004" [9] "H\024?\001\f\025?\001,??\001\002\002" [10] "?\026?\001?$?\001\005" [11] "" [12] "s.logical" ... [75] "8}?\001?~?\001\020??\001?$?\001?}?\001?$?\001???\001 $?\001???\001?~?\0014~?\001???\001?|?\001???\001???\001\034}?\001???\001 $?\001?|?\001?\177?\001" [76] "?$?\001?$?\001???\001\030??\001T??\001?$?\001?$?\001 $?\001???\0010??\001?$?\001?$?\001???\0014??\001 \177?\001X??\001\034??\001?$?\001???\001???\001 |?\001\004??\001???\0010\177?\001???\001?{?\001<??\001 $?\001???\001?}?\001 ??\001?$?\001?$?\001\024\177?\001 $?\001t??\001???\001?}?\001?$?\001?$?\001t|?\001?$?\001 $?\001?\177?\001???\001?|?\001 |?\001?$?\001d??\001 $?\001???\001x{?\001?$?\001" [77] "' must be of length 1" This behavior has been noticed by me at least as early as 2.6, and is currently seen under 2.9.0. Thanks for any pointers, Jeff Ryan -- Jeffrey Ryan jeffrey.ryan at insightalgo.com ia: insight algorithmics www.insightalgo.com
Gabor Grothendieck
2009-May-18 18:01 UTC
[Rd] readBin on binary non-blocking connections (Windows & Unix differences/bugs)
Ryacas uses non-blocking sockets and works across all platforms but uses readLines/writeLines, rather than readBin, to communicate with yacas. You could look at its source code in case it brings anything to mind. On Mon, May 18, 2009 at 1:40 PM, Jeff Ryan <jeff.a.ryan at gmail.com> wrote:> R-devel: > > I am encountering a consistency issue using socketConnection and > readBin with *non-blocking* connections on Unix and Windows XP (no > Vista to test). > > I am a bit confused by the behavior of *non-blocking* connections > under Windows specifically. ?When calling readBin on a non-blocking > connection when there is no data to read on the socket, the connection > under Unix will return a vector of zero-length matching the type of > the 'what' argument to the readBin call. > > The same call under Windows returns random data unless called with > what=raw(), which instead returns an error. > > > A simple example using two R processes to illustrate on one machine: > > > ######################################################################### > # PROCESS 1 (Windows/Unix -- the same code for both): > ######################################################################### > > # Open the connection and don't write anything > > s <- socketConnection(port=7777, server=TRUE, blocking=TRUE,open="ab") > > ######################################################################### > # PROCESS 2 (Windows) use this to read: > ######################################################################### > > s <- socketConnection(port=7777,blocking=FALSE,open="ab") > readBin(s, character()) > readBin(s, character()) > readBin(s, double()) > readBin(s, raw()) > > # > readBin(s, character()) > # [1] "\020??\001@??\001 $?\001 $?\001 $?\001 $?\001 $?\001 $?\001 > $?\001 $?\001 ... > # > readBin(s, character()) > # [1] "X\n" > # > readBin(s, double()) > # [1] 1.255609e-316 > # > readBin(s, raw()) > # Error in readBin(s, raw()) : negative length vectors are not allowed > > ########################################################################## > # Using a *nix, the above works as I would expect > # > # PROCESS 2 (Unixes -- this example OSX 10.4, but is consistent on > other flavors) > ######################################################################### > > s <- socketConnection(port=7777,blocking=FALSE, open="ab") > readBin(s, character()) > readBin(s, raw()) > readBin(s, double()) > > # > readBin(s, character()) > # character(0) > # > readBin(s, raw()) > # raw(0) > # > readBin(s, double()) > # numeric(0) > > Is this a bug in R or Windows? ?Is there a workaround? ?I would like > to have readBin on non-blocking connections work consistently across > platforms. ?When data is present, both platforms behave consistently. > Incidentally, the readBin calls in my program are not using R as the > server, the above code is simply to illustrate the problem. > > The fact that the raw() returns an error on Windows seems to indicate > that the error is able to be caught, and could be tested for. > > One other interesting point of note; the character() calls on Windows > will return not just random data, but very non-random R type strings. > Somehow non-allocated memory is being read. > >> readBin(s, character(),100) > ?[1] "X\n" > ?[2] "spatch" > ?[3] "" > ?[4] "" > ?[5] "X\n" > ?[6] "ssion" > ?[7] "\002" > ?[8] "?$?\001?|?\001\030[?\001?N?\001?N?\001?$?\001\004" > ?[9] "H\024?\001\f\025?\001,??\001\002\002" > ?[10] "?\026?\001?$?\001\005" > ?[11] "" > ?[12] "s.logical" > ... > ?[75] "8}?\001?~?\001\020??\001?$?\001?}?\001?$?\001???\001 > $?\001???\001?~?\0014~?\001???\001?|?\001???\001???\001\034}?\001???\001 > $?\001?|?\001?\177?\001" > ?[76] "?$?\001?$?\001???\001\030??\001T??\001?$?\001?$?\001 > $?\001???\0010??\001?$?\001?$?\001???\0014??\001 > \177?\001X??\001\034??\001?$?\001???\001???\001 > |?\001\004??\001???\0010\177?\001???\001?{?\001<??\001 > $?\001???\001?}?\001 ??\001?$?\001?$?\001\024\177?\001 > $?\001t??\001???\001?}?\001?$?\001?$?\001t|?\001?$?\001 > $?\001?\177?\001???\001?|?\001 |?\001?$?\001d??\001 > $?\001???\001x{?\001?$?\001" > ?[77] "' must be of length 1" > > This behavior has been noticed by me at least as early as 2.6, and is > currently seen under 2.9.0. > > Thanks for any pointers, > Jeff Ryan > > > -- > Jeffrey Ryan > jeffrey.ryan at insightalgo.com > > ia: insight algorithmics > www.insightalgo.com > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel >
Jeff Ryan
2009-May-19 19:26 UTC
[Rd] readBin on binary non-blocking connections (Windows & Unix differences/bugs)
R-devel: I'll escalate this to a bug report once I can fully document, but something is seriously wrong with readBin on non-blocking connections.>From ?readBinValue: For 'readBin', a vector of appropriate mode and length the number of items read (which might be less than 'n'). On Windows, 'n' elements are always returned. If 'n' < items waiting to be read, then the vector of length 'n' will be returned with the expected items. If n > items, a vector of length n (yes, n) will be returned, padded with random values taken from (somewhere???) in memory. As expressed in a previous email, if what=raw() and n > items, the entire process fails with an error. The items are effectively read and discarded. I would be interested in assisting further with a patch, but I would hope that someone (R-core or otherwise) who knows a bit more about the internals of socket connections in R (w.r.t Windows - as *nix works fine) could provide some much needed insight. Best, Jeff Ryan On Mon, May 18, 2009 at 12:40 PM, Jeff Ryan <jeff.a.ryan at gmail.com> wrote:> R-devel: > > I am encountering a consistency issue using socketConnection and > readBin with *non-blocking* connections on Unix and Windows XP (no > Vista to test). > > I am a bit confused by the behavior of *non-blocking* connections > under Windows specifically. ?When calling readBin on a non-blocking > connection when there is no data to read on the socket, the connection > under Unix will return a vector of zero-length matching the type of > the 'what' argument to the readBin call. > > The same call under Windows returns random data unless called with > what=raw(), which instead returns an error. > > > A simple example using two R processes to illustrate on one machine: > > > ######################################################################### > # PROCESS 1 (Windows/Unix -- the same code for both): > ######################################################################### > > # Open the connection and don't write anything > > s <- socketConnection(port=7777, server=TRUE, blocking=TRUE,open="ab") > > ######################################################################### > # PROCESS 2 (Windows) use this to read: > ######################################################################### > > s <- socketConnection(port=7777,blocking=FALSE,open="ab") > readBin(s, character()) > readBin(s, character()) > readBin(s, double()) > readBin(s, raw()) > > # > readBin(s, character()) > # [1] "\020??\001@??\001 $?\001 $?\001 $?\001 $?\001 $?\001 $?\001 > $?\001 $?\001 ... > # > readBin(s, character()) > # [1] "X\n" > # > readBin(s, double()) > # [1] 1.255609e-316 > # > readBin(s, raw()) > # Error in readBin(s, raw()) : negative length vectors are not allowed > > ########################################################################## > # Using a *nix, the above works as I would expect > # > # PROCESS 2 (Unixes -- this example OSX 10.4, but is consistent on > other flavors) > ######################################################################### > > s <- socketConnection(port=7777,blocking=FALSE, open="ab") > readBin(s, character()) > readBin(s, raw()) > readBin(s, double()) > > # > readBin(s, character()) > # character(0) > # > readBin(s, raw()) > # raw(0) > # > readBin(s, double()) > # numeric(0) > > Is this a bug in R or Windows? ?Is there a workaround? ?I would like > to have readBin on non-blocking connections work consistently across > platforms. ?When data is present, both platforms behave consistently. > Incidentally, the readBin calls in my program are not using R as the > server, the above code is simply to illustrate the problem. > > The fact that the raw() returns an error on Windows seems to indicate > that the error is able to be caught, and could be tested for. > > One other interesting point of note; the character() calls on Windows > will return not just random data, but very non-random R type strings. > Somehow non-allocated memory is being read. > >> readBin(s, character(),100) > ?[1] "X\n" > ?[2] "spatch" > ?[3] "" > ?[4] "" > ?[5] "X\n" > ?[6] "ssion" > ?[7] "\002" > ?[8] "?$?\001?|?\001\030[?\001?N?\001?N?\001?$?\001\004" > ?[9] "H\024?\001\f\025?\001,??\001\002\002" > ?[10] "?\026?\001?$?\001\005" > ?[11] "" > ?[12] "s.logical" > ... > ?[75] "8}?\001?~?\001\020??\001?$?\001?}?\001?$?\001???\001 > $?\001???\001?~?\0014~?\001???\001?|?\001???\001???\001\034}?\001???\001 > $?\001?|?\001?\177?\001" > ?[76] "?$?\001?$?\001???\001\030??\001T??\001?$?\001?$?\001 > $?\001???\0010??\001?$?\001?$?\001???\0014??\001 > \177?\001X??\001\034??\001?$?\001???\001???\001 > |?\001\004??\001???\0010\177?\001???\001?{?\001<??\001 > $?\001???\001?}?\001 ??\001?$?\001?$?\001\024\177?\001 > $?\001t??\001???\001?}?\001?$?\001?$?\001t|?\001?$?\001 > $?\001?\177?\001???\001?|?\001 |?\001?$?\001d??\001 > $?\001???\001x{?\001?$?\001" > ?[77] "' must be of length 1" > > This behavior has been noticed by me at least as early as 2.6, and is > currently seen under 2.9.0. > > Thanks for any pointers, > Jeff Ryan > > > -- > Jeffrey Ryan > jeffrey.ryan at insightalgo.com > > ia: insight algorithmics > www.insightalgo.com >-- Jeffrey Ryan jeffrey.ryan at insightalgo.com ia: insight algorithmics www.insightalgo.com