Ramon Diaz-Uriarte
2008-Feb-29 18:16 UTC
[R] using zlib (was compress data on read, decompress on write)
Dear All, I think I am confused about how I'd be able to use zlib starting with R 2.7.0. I just downloaded the latest development version, built it, etc, but I am not able to find the zlib.h that, I believe, R should place somewhere under "where/you/want/R/to/go" (from --prefix=/where/you/want/R/to/go). There seem to be entry points for zlib in the binary (e.g., grep -r gzopen ./ Binary file ./lib64/R/bin/exec/R matches; etc). What is the appropriate way (if any) to tell my package where to look for the R-provided zlib.h? In addition, when searching the archives I found the message http://finzi.psych.upenn.edu/R/R-devel/archive/27154.html which seems to suggest that I'd be better off including my own copies of zlib.h, although I understand that the message is not referring explicitly to the new R 2.7.0. Thanks, R. On Thu, Feb 28, 2008 at 8:53 PM, Prof Brian Ripley <ripley at stats.ox.ac.uk> wrote:> One solution is likely to be the Omegahat package Rcompression. > > Otherwise, R does have internal facilities to do internal (gzip) > compression and decompression (e.g. see the end of > src/main/connections.c), and you could make creative use of serialization > to do the compression. > > > On Thu, 28 Feb 2008, Ramon Diaz-Uriarte wrote: > > > Dear All, > > > > I'd like to be able to have R store (in a list component) a compressed > > data set, and then write it out uncompressed. gzcon and gzfile work in > > exactly the opposite direction. What would be a good way to handle > > this? > > > > Details: > > ---------- > > > > We have a package that uses C; part of the C output is a large sparse > > matrix. This is never manipulated directly by R, but always by the C > > code. However, we need to store that data somewhere (inside an R > > object) for further calls to the functions in our package. We'd like > > to store that matrix as part of the R object (say, as an element of a > > list). Ideally, it would be stored in as compressed a way as possible. > > Then, when we need to use that information, it would be decompressed > > and passed to the C function. > > > > I guess one way to do it is to have C deal with the compression and > > uncompression (e.g., using zlib or the bzip2 libraries) and then use > > readBin, etc, from R. But, if I can, I'd like to avoid our C code > > having to call zlib, etc, so as to make our package easily portable. > > As from R 2.7.0 you will be able to make use of zlib on effectively all > platforms, since it has a public interface on Windows. > > > > > > > Thanks, > > > > R. > > > > -- > > Ramon Diaz-Uriarte > > Statistical Computing Team > > Structural Biology and Biocomputing Programme > > Spanish National Cancer Centre (CNIO) > > http://ligarto.org/rdiaz > > -- > Brian D. Ripley, ripley at stats.ox.ac.uk > Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ > University of Oxford, Tel: +44 1865 272861 (self) > 1 South Parks Road, +44 1865 272866 (PA) > Oxford OX1 3TG, UK Fax: +44 1865 272595 >-- Ramon Diaz-Uriarte Statistical Computing Team Structural Biology and Biocomputing Programme Spanish National Cancer Centre (CNIO) http://ligarto.org/rdiaz
Prof Brian Ripley
2008-Feb-29 18:36 UTC
[R] using zlib (was compress data on read, decompress on write)
On Fri, 29 Feb 2008, Ramon Diaz-Uriarte wrote:> Dear All, > > I think I am confused about how I'd be able to use zlib starting with > R 2.7.0. I just downloaded the latest development version, built it, > etc, but I am not able to find the zlib.h that, I believe, R should > place somewhere under "where/you/want/R/to/go" (from > --prefix=/where/you/want/R/to/go). There seem to be entry points for > zlib in the binary (e.g., grep -r gzopen ./ > Binary file ./lib64/R/bin/exec/R matches; etc). What is the > appropriate way (if any) to tell my package where to look for the > R-provided zlib.h?It is only provided on Windows -- that's the only change. On other platforms you use configure to find the system copy in the usual way. The point is that almost all platforms apart from Windows will have a systen copy. What you need to do on Windows is in the CHANGES file. [You may find entry points in exec/R, but those using shlib R will not.]> In addition, when searching the archives I found the message > > http://finzi.psych.upenn.edu/R/R-devel/archive/27154.html > > which seems to suggest that I'd be better off including my own copies > of zlib.h, although I understand that the message is not referring > explicitly to the new R 2.7.0.Nor implicitly.> > Thanks, > > R. > > > > > > > > On Thu, Feb 28, 2008 at 8:53 PM, Prof Brian Ripley > <ripley at stats.ox.ac.uk> wrote: >> One solution is likely to be the Omegahat package Rcompression. >> >> Otherwise, R does have internal facilities to do internal (gzip) >> compression and decompression (e.g. see the end of >> src/main/connections.c), and you could make creative use of serialization >> to do the compression. >> >> >> On Thu, 28 Feb 2008, Ramon Diaz-Uriarte wrote: >> >> > Dear All, >> > >> > I'd like to be able to have R store (in a list component) a compressed >> > data set, and then write it out uncompressed. gzcon and gzfile work in >> > exactly the opposite direction. What would be a good way to handle >> > this? >> > >> > Details: >> > ---------- >> > >> > We have a package that uses C; part of the C output is a large sparse >> > matrix. This is never manipulated directly by R, but always by the C >> > code. However, we need to store that data somewhere (inside an R >> > object) for further calls to the functions in our package. We'd like >> > to store that matrix as part of the R object (say, as an element of a >> > list). Ideally, it would be stored in as compressed a way as possible. >> > Then, when we need to use that information, it would be decompressed >> > and passed to the C function. >> > >> > I guess one way to do it is to have C deal with the compression and >> > uncompression (e.g., using zlib or the bzip2 libraries) and then use >> > readBin, etc, from R. But, if I can, I'd like to avoid our C code >> > having to call zlib, etc, so as to make our package easily portable. >> >> As from R 2.7.0 you will be able to make use of zlib on effectively all >> platforms, since it has a public interface on Windows. >> >> >> >> > >> > Thanks, >> > >> > R. >> > >> > -- >> > Ramon Diaz-Uriarte >> > Statistical Computing Team >> > Structural Biology and Biocomputing Programme >> > Spanish National Cancer Centre (CNIO) >> > http://ligarto.org/rdiaz >> >> -- >> Brian D. Ripley, ripley at stats.ox.ac.uk >> Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ >> University of Oxford, Tel: +44 1865 272861 (self) >> 1 South Parks Road, +44 1865 272866 (PA) >> Oxford OX1 3TG, UK Fax: +44 1865 272595 >> > > > > -- > Ramon Diaz-Uriarte > Statistical Computing Team > Structural Biology and Biocomputing Programme > Spanish National Cancer Centre (CNIO) > http://ligarto.org/rdiaz >-- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595