search for: tspace

Hello, We have created an interface between R and Hadoop so that the user can, after a fashion, interact with very large datasets using the Map Reduce programming model. We also use IBM's TSpaces to implement a shared memory implementation that can be accessed via R(somewhat like networkspaces). RHIPE uses Rserve to execute R code. Some of the functions implemented are: mrlapply - run lapply across a Hadoop cluster mrsubsetf - subset a file according to an R function mtapplyf - run a...

...ines and collect side effect files. Since RHIPE uses Hadoop for distributing computation, it also benefits from Hadoops stability: load balancing and machine failure recovery being two important features, scheduling of jobs etc. RHIPE also implements a basic Shared Associative Space via IBM's TSpaces - with commands like rhput,rhtake,rhread etc. This is optional. See the website for details and performance results. Unfortunately, I haven't had the chance to compare with rmpi and snow. TODO(will appear soon ): another function: rhmr - mapreduce using R. Regards Saptarshi Guha [[alterna...