R devel - Jun 2000 - Binary file functions in R

This is just to let people know that I'm going to try to put together
a little package of binary file read/write functions for R, along the
lines described in my message in r-help.  I'm going to do it first in
R for Windows using a DLL written in Delphi; I'll let people comment
on that, and once the design settles down, I'll redo the external code
in C so that it should be portable to Unix as well.

One immediate change from my first pass suggestion below:

I'll just have readint, readfloat, readchar; the size and whether
signed or unsigned will be determined by parameters to those
functions.

Other suggested changes are welcome!

Duncan Murdoch
>I think a set of stream like functions would be really handy
>sometimes, especially for those of us who aren't that comfortable with
>the external function interface.
>
>For example:
>
>fopen would take a filename and some information about the open mode
>(read and write would probably be enough modes) and would return a
>file handle.
>
>readint8, readint16, readint32, readint64 would read a specified count
>of integers into an integer vector.
>
>readfloat4, readfloat8 would read a specified count of floating point
>values into a vector of doubles.
>
>readchar would read a specified count of characters into a single
>component character vector, or take two counts, and read n characters
>m times into an m component vector.
>
>There would be similar writexxxx functions, a seek function, and an
>fclose function.
>
>Some issues to consider would be whether these functions should worry
>about big vs. little endian storage (I'd say probably not, but there
>should be other functions that can change it in memory), how errors
>should be handled, whether the garbage collector should be smart
>enough to automatically close an abandoned file, whether you should be
>allowed to use these low level functions on things in memory or just
>on files, etc.
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To:
r-devel-request@stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._

I've put together the first version of "Rstreams", a package that
allows binary access to files.  This version is for Windows only.
You can pick it up as
<http://www.stats.uwo.ca/faculty/murdoch/software/rstreams.zip>.
Comments are welcome!

Some known problems:

I tried to set it up as a package, but couldn't figure out from the
documentation where the .DLL should go.  It's sitting in the src
directory for now.

There are other things wrong with the organization as a package; R
says "no R code found" when I try to attach it.  If you want to use
it, you'll have to source the Rstreams.R file and dyn.load the DLL
yourself.

I used the .C() interface to external functions, but I'm not sure how
it handles character vectors being returned from DLLs.  I'm sticking
string pointers into the result vector, which is likely to cause
memory problems, since I don't use Ralloc to allocate the strings.
How are you supposed to return a string value?  (This only applies to
the print method; it needs to find out the filename.  The readchar
method pre-allocates the space that's needed and uses that.)

The routines that I've implemented so far are:

openstream
closestream
print.stream
readint
readfloat
readchar
writeint
writefloat
writechar
seek
truncate

One other comment for Robert G.::

Most of the information about a stream variable is kept in the DLL.
All that R gets is a handle.  This means you're safe to make
assignments of stream variables to other variables; both refer to the
same stream object in the DLL.  That's how I think environments should
be handled.

Again, comments are welcome!

Duncan Murdoch
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To:
r-devel-request@stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._