Displaying 20 results from an estimated 2000 matches similar to: "bigmemory and R 3.0"
2012 May 05
2
looking for adice on bigmemory framework with C++ and java interoperability
I work with problems that have rather large data requirements -- typically
a bunch of multigig arrays. Given how generous R is with using memory, the
only way for me to work with R has been to use bigmatrices from bigmemory
package. One thing that is missing a bit is interoperability of bigmatrices
with C++ and possibly java. What i mean by that is API that would allow
read and write filebacked
2010 Aug 11
1
Bigmemory: Error Running Example
Hi,
I am trying to run the bigmemory example provided on the
http://www.bigmemory.org/
The example runs on the "airline data" and generates summary of the csv
files:-
library(bigmemory)
library(biganalytics)
x <- read.big.matrix("2005.csv", type="integer", header=TRUE,
backingfile="airline.bin",
descriptorfile="airline.desc",
2010 May 10
0
bigmemory 4.2.3
The long-promised revision to bigmemory has arrived, with package
4.2.3 now on CRAN. The mutexes (locks) have been extracted and will
be available through package synchronicity (on R-Forge, soon to appear
on CRAN). Initial versions of packages biganalytics and bigtabulate
are on CRAN, and new versions which resolve the warnings and have
streamlined CRAN-friendly configurations will appear
2010 May 10
0
bigmemory 4.2.3
The long-promised revision to bigmemory has arrived, with package
4.2.3 now on CRAN. The mutexes (locks) have been extracted and will
be available through package synchronicity (on R-Forge, soon to appear
on CRAN). Initial versions of packages biganalytics and bigtabulate
are on CRAN, and new versions which resolve the warnings and have
streamlined CRAN-friendly configurations will appear
2010 Jun 15
1
help biglm.big.matrix; problem with weights
Hello colleagues,
I have tried to use the package biglm. I want to specify a
multivariate regression with a weight.
I have imported a large dataset with the library(bigmemory). I load
the library (biglm) and specified a regression with a weight. But I
get everytime a error message like ?object not found? or ?`weights'
must be a formula? or "error in eval(expr, envir, enclos)". I
2011 Nov 30
1
Error adding Bigmemory package
I am trying to add the bigmemory packages but I get the following error
message:
In file included from bigmemory.cpp:14:0:
../inst/include/bigmemory/isna.hpp: In function 'bool neginf(double)':
../inst/include/bigmemory/isna.hpp:22:57: error: 'isinf' was not declared in
this scope
make: *** [bigmemory.o] Error 1
ERROR: compilation failed for package 'bigmemory'
* removing
2011 Jun 24
1
Installation of bigmemory fails
Hello All,
I tried to intall the bigmemory package from a CRAN mirror site and
received the following output while installing. Any idea what's going
on and how to fix it? The system details are provided below.
--------------------- begin error messages -----------------------
* installing *source* package 'bigmemory' ...
checking for Sun Studio compiler...no
checking for
2011 Feb 11
2
Large Datasets
I have recently been using R - more speciifcally the GUI packages Rattle
and Rcmdr.
I like these products a lot and want to use them for some projects - the problem
that I run into is when I start to try and run large datasets through them. The
data sets are 10-15 million in record quantity and usually have 15-30 fields
(both numerical and categorical).
I saw that there were some packages
2010 Feb 22
1
big panel: filehash, bigmemory or other
Dear R-list
I'm on my way to start a new project on a rather big panel, consisting
of approximately 8 million observations in 30 waves of data and about
15 variables. I have a similar data set that is approximately 7
gigabytes in size.
Until now I have done my data management in SAS, and Stata, mostly
identifying spells, counting events in intervals, and a like, but I
would like to
2012 May 11
1
bigmemory
To answer your first question about read.big.matrix(), we don't know what
your acc3.dat file is, but it doesn't appear to have been detected as a
standard file (like a CSV file) or -- perhaps -- doesn't even exist (or
doesn't exist in your current directory)?
Next:
> In addition, I am planning to do a multiple imputation with MICE package
> using the data read by bigmemory
2019 Jul 19
1
difficulty with sanitizer using bigmemory
Dear all,
bigKRLS, which has been on CRAN for a couple of years, had to be pulled
recently due to what seems to be a sanitizer issue stemming from its use of
bigmemory. bigKRLS works fine (we?ve used it ourselves on many different
platforms and have had over 15,000 downloads without an end user reporting
difficulties because of this issue). Unfortunately, we have been unable to
reproduce the
2011 Sep 29
1
efficient coding with foreach and bigmemory
I recently learned about the bigmemory and foreach packages and am trying
to use them to help me create a very large matrix. Without those
packages, I can create the type of matrix that I want with 10 columns and
5e6 rows. I would like to be able to scale up to 5e9 rows, or more, if
possible.
I have created a simplified example of what I'm trying to do, below. The
first part of the
2010 Dec 17
1
[Fwd: adding more columns in big.matrix object of bigmemory package]
Hi,
With reference to the mail below, I have large datasets, coming from various
different sources, which I can read into filebacked big.matrix using library
bigmemory. I want to merge them all into one 'big.matrix' object. (Later, I
want to run regression using library 'biglm').
I am unsuccessfully trying to do this from quite some time now. Can you
please
2010 Apr 23
2
bigmemory package woes
I have pretty big data sizes, like matrices of .5 to 1.5GB so once i need to
juggle several of them i am in need of disk cache. I am trying to use
bigmemory package but getting problems that are hard to understand. I am
getting seg faults and machine just hanging. I work by the way on Red Hat
Linux, 64 bit R version 10.
Simplest problem is just saving matrices. When i do something like
2017 Jan 09
4
Tweaking the Register Allocator's spill placement
Hello,
My target features some very-high-latency instructions that access an on-chip network (we'll call them FXLV). In one important kernel (snippet below), register allocation needs to spill values resulting from FXLV. The spiller is unaware of FXLV's latency, and thus naively inserts those spills immediately after the FXLV, incurring huge and unnecessary data stalls.
FXLV r10,
2010 Sep 08
1
bigmemory doubt
Hi,
Is it possible for me to read data from shared memory created by a vc++ program into R using bigmemory?
[[alternative HTML version deleted]]
2012 Jan 18
1
kmeans clustering on large but sparse matrix
Hi,
I have a 60k*600k matrix, which exceed the vector length limit of 2^32-1.
But it's rather sparse, only 0.02% has value. So I save is as MarketMatrix
(mm) file, it's about 300M in size. I use readMM in Matrix package to read
it in. If do so, the data type becomes dgTMatrix in 'Matrix' package
instead of the common matrix type.
The problem is, if I run k-means only on part of
2011 Dec 01
1
bigmemory on Solaris
At one point we might have gotten something working (older version?) on
Solaris x86, but were never successful on Solaris sparc that I remember --
it isn't a platform we can test and support. We believe there are problems
with BOOST library compatibilities.
We'll try (again) to clear up the other warnings in the logs, though. !-)
We should also revisit the possibility of a CRAN BOOST
2012 Oct 18
3
bigmemory for dataframes?
Hi Folks,
I've been bumping my head against the 4GB limit for 32-bit R. I can't
go to 64-bit R due to package compatibility issues (ROBDC - possible but
painful, xlsReadWrite - not possible, and others). I have a number of
big dataframes whose columns all sorts of data types - factor,
character, integer, etc. I run and save models that keep copies of the
modeled data inside the model
2013 Jun 27
2
choicemodelr is misbehaving under R3.0
Hi John (and other authors of ChoiceModelR package),
I am experiencing a weird thing when using the function choicemodelr under
R3.0.1.
Before I updated to R3.0, I had used choicemodelr unde R2.15. It was always
as fast or faster than exactly the same task in Sawtooth software.
Now, I've tried to run it under R3.0.1. It seems to be doing the job. But
something is slowing it down dramatically.