R packages - Nov 2012 - package bit64 with new functionality

Dear R community,

The new version of package 'bit64' - which extends R with fast 64-bit 
integers - now has fast (single-threaded) implementations of the most 
important univariate algorithmic operations (those based on hashing and 
sorting). Package 'bit64' now has methods for 'match',
'%in%',
'duplicated', 'unique', 'table', 'sort',
'order', 'rank', 'quantile',
'median' and 'summary'. Regarding data management it has novel
generics
'unipos' (positions of the unique values), 'tiepos' (positions
of ties),
'keypos' (positions of values in a sorted unique table) and derived 
methods 'as.factor' and 'as.ordered'. This 64-bit functionality
is
implemented carefully to be not slower than the respective 32-bit 
operations in Base R and also to avoid excessive execution times 
observed with 'order', 'rank' and 'table' (speedup
factors 20/16/200
respective). This increases the dataset size with wich we can work truly 
interactive. The speed is achieved by simple heuristic optimizers: the 
mentioned high-level functions choose the best from multiple low-level 
algorithms and further take advantage of a novel optional caching 
method. In an example R session using a couple of these operations the 
64-bit integers performed 22x faster than base 32-bit integers, 
hash-caching improved this to 24x amortized, sortorder-caching was most 
efficient with 38x (caching both, hashing and sorting is not worth it 
with 32x at duplicated RAM consumption).

Since the package covers the most important functions for (univariate) 
data exploration and data management, I think it is now appropriate to 
claim that R has sound 64-bit integer support, for example for working 
with keys or counts imported from large databases. For details 
concerning approach, implementation and roadmap please check the 
ANNOUNCEMENT-0.9-Details.txt file and the package help files.

Kind regards


Jens Oehlschl?gel
Munich, 8.11.2012

_______________________________________________
R-packages mailing list
R-packages at r-project.org
https://stat.ethz.ch/mailman/listinfo/r-packages