similar to: R as analysis server for very large data sets

One way to deal with this is to install R itself with mpicc. Then all packages are installed with mpicc and get the required MPI libraries and includes by default. I have done this with R-2.13.0 on an Opteron cluster running CentOS-5 a while ago and so far it has worked out great. I crosspost to r-sig-hpc and welcome others to comment if there are potential downsides to this solution. George

Dear R Users, How do we read netcdf / hdf format files in R ? Also, can we convert netcdf to hdf format in R? Great Thanks, Best Regards, Yogesh [[alternative HTML version deleted]]

Hi, I've been playing with R for only a few days, so I'm not terribly familiar with it yet. Has anyone ever done any work using HDF or NetCDF files with R? Both file formats have libraries which can be linked to a program to read and write data to these formats. Is this sort of thing possible with R? We are planning to use the HDF format to store experimental data collected by some

If a move toward machine-independent representation seems reasonable, it may be worth looking at some of the binary data exchange formats used by nasa and others, CDF http://nssdc.gsfc.nasa.gov/cdf/cdf_home.html netCDF http://www.unidata.ucar.edu/packages/netcdf/index.html HDF http://hdf.ncsa.uiuc.edu/ There may also be something useful in the "Scientific Data Format Information

I am compiling the Embedding examples in the tests directory and get an undefined reference. I include the make output as well as grep'd output of nm on libR.so and compiler and arch information. Do I have an improperly built R shared library or is there a problem with the Embedding tests or something else I am not seeing? Thanks for any help! George ost at

Hi, I saw some ref to R on HDF site and found some e-mails regarding them on Feb. Has there ben any developement on either of these software (HDF or netCDF)? I am primary interested in NT side, but Unix side will help to some degree. Thanks much in advance, ************************************************************** S. Hoon Yoon (Quant) Merrill Lynch

I want to do a simple histogram for a very large data set. I traced the problem to the range() function and the storage.mode()<-"double" assignment. Below is a sequence that works for min() and max(), but fails for range(). > length(evap0) [1] 51812400 > mode(evap0) [1] "numeric" > storage.mode(breaks) [1]

Should one ask questions relating to the R package 'ncdf' here? or look for a more netCDF-oriented (but probably less R-oriented) list? Why I ask: I'm relatively new to R, which I've only used in the past for graphics. I'm trying to learn R as an alternative to manipulating netCDF files with NCO (which works well but lacks all the other R goodnesses). Hence I've got a

Has anyone experimented with the Network Common Data Format (netcdf) described at http://www.unidata.ucar.edu/packages/netcdf for archiving data? Is this format widely used outside of atmospheric research? -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info",

In response to the Aug 20 posting by Vadim, I had the same issue with all kinds of complaints about row.names when it was not given in the read.table command; when it was set to NULL (data were actually shifted to the wrong names). I believe the problem was no comma (or specified delimiter) at the end of the first row with the names in it. Don't know why, but in comparing two programs,

This is an automated email from the git hooks/post-receive script. unknown user pushed a commit to branch master in repository centos/centos.org. The following commit(s) were added to refs/heads/master by this push: new c10f929 ornl video c10f929 is described below commit c10f92988b176fab93f5f37f0f37f39c654c180f Author: rbowen <rbowen at rcbowen.com> AuthorDate: Mon May 20 09:09:43

Hi! I am trying to install in my R environment the rhdf5 package and library but it seems to have vanished from either the CRAN or BioConductors sites. Can you tell me where it would be possible to find it or any R library (or function) able to read hdf files? Sincerely, Nicolas Degallier UMR 7159 / IRD UR182 Laboratoire d'Oc?anographie et du Climat, Exp?rimentation et Approches

Hi all I'm happy to announce a new version of wbstats is now available on CRAN. wbstats is an R package for searching and downloading data from the World Bank API that includes access to all annual, monthly, and quarterly indicators in addition to bug fixes. Version 0.2 now * Uses version 2 of the World Bank API. This includes data that is unavailable in the previous world bank API *

Hi all I'm happy to announce a new version of wbstats is now available on CRAN. wbstats is an R package for searching and downloading data from the World Bank API that includes access to all annual, monthly, and quarterly indicators in addition to bug fixes. Version 0.2 now * Uses version 2 of the World Bank API. This includes data that is unavailable in the previous world bank API *

Per the announcement from the MVAPICH team, I am pleased to let you know that the MPI-IO support for Lustre has been integrated into the new release of MVAPICH, version 1.0. > - Optimized and high-performance ADIO driver for Lustre > - This MPI-IO support is a contribution from Future Technologies > Group, Oak Ridge National Laboratory. >

Dear All, I have data in HDF file format and would like to read it into R. I have tried the package hdf5 without success. Any ideas and suggestions?? Kind regards, Katrin -- Katrin Fleischer Vrije Universiteit Amsterdam Faculty of Earth and Life Sciences Subdepartment Hydrolgy and Geo-Environmental Sciences Room E-360 De Boelelaan 1085 1081 HV AMSTERDAM Tel: +31 20 59 87391

I got a netCDF file from a climate group and wanted to aggregate the data. The file summary looks like this library(ncdf) nc=open.ncdf("NCEP_prec_1949_58.nc") > nc [1] "file NCEP_prec_1949_58.nc has 4 dimensions:" [1] "Time Size: 14608" [1] "lat Size: 99" [1] "lon Size: 199" [1] "DateStrLen Size: 20" [1]

Dear R community, I find R fantastic and use R whenever I can for my data analytic needs. Certain data sets, however, are so large that other tools seem to be needed to pre-process data such that it can be brought into R for further analysis. Questions I have for the many expert contributors on this list are: 1. How do others handle situations of large data sets (gigabytes, terabytes)

hello, I have a problem to open an hdf file. i have downloaded the package 'hdf5' as it was advised on R seek. But when i try to load the file, the R console sends me an eror message: setwd("C:/Documents and Settings/Karine/Bureau/data/") #install.packages('hdf5') library("hdf5") sea_ice <- hdf5load("asi-s6250-20090704-v5i.hdf", load =

Hi All, I am using the allShortestPath function based on Floyd's algorithm in e1071 package. It runs great when I have less than 5000 nodes. But when I tried to work on more than 5000 nodes, I ran into memory problem. The problem I really want to solve has 10000-15000 nodes. Does anybody know how to deal with this problem? Are there any other packages in R that can handle this problem?