similar to: Memory issues

Hi everyone, I’m trying to install reshape2. But when I click on “install package” it’s not coming up!?!?! I’m getting reshape, but no reshape2? I’ve also tried download.packages(reshape2, destdir="c:\\") & download.packages(Reshape2, destdir="c:\\")…but no luck!!! Does anyone have any ideas what could be going on? Chris Howden Founding Partner Tricky

Hi everyone, I’m working with some very big datasets (each dataset has 11 million rows and 2 columns). My first step is to merge all my individual data sets together (I have about 20) I’m using the following command from sqldf data1 <- sqldf("select A.*, B.* from A inner join B using(ID)") But it’s taking A VERY VERY LONG TIME to merge just 2 of the datasets

Hi everyone, I’m looking for a clever bit of code to replace NA’s with a specific score depending on an indicator variable. I can see how to do it using lots of if statements but I’m sure there most be a neater, better way of doing it. Any ideas at all will be much appreciated, I’m dreading coding up all those if statements!!!!! My problem is as follows: I have a data set with

Hi everyone, I’ve recently added a private library as a way to manage my R libraries. And I did this by simply copying my old library to a new folder and then linking this to R by setting my R_LIBS environmental variable in .Renviron. However I have run into a problem. When I update my packages it is not updating those that are current in the base R library. This means I can’t load

Dear all, bigKRLS, which has been on CRAN for a couple of years, had to be pulled recently due to what seems to be a sanitizer issue stemming from its use of bigmemory. bigKRLS works fine (we?ve used it ourselves on many different platforms and have had over 15,000 downloads without an end user reporting difficulties because of this issue). Unfortunately, we have been unable to reproduce the

Hi all, I have a vector that contains the row numbers of data taken from several filtering operations performed on a large data frame (20,000rows x 500cols). In order to output this subset of data, I've been looping through the vector containing the row numbers (keepRows). output <- data.frame(row.names = rownames(bigMatrix)) for(i in keepRows) { output <- rbind(output,

Hello I'm a population ecologist and use R for all my stats and modelling. Recently I have been using R to numerically solve integral projection models. This involves constructing several large matrices. The current code by Easterling (Size-specific sensitivity: Applying a new structured population model. Ecology, 2000, 81, 694-708) uses nested loops to construct the matrices. To speed up the

Hi, I am trying to do multinomial regression using the glmnet package, but the following gives me an error (for no reason apparent to me): library(glmnet) cv.glmnet(x=matrix(c(1,2,3,4,5,6,1,2,3,4,5,6), nrow=6),y=as.factor(c(1,2,1,2,3,3)),family='multinomial',alpha=0.5, nfolds=2) The error i get is: Error in if (outlist$msg != "Unknown error") return(outlist) : argument is of

We are building a package, and want to create S4 methods for both head and mean for our own BigMatrix class. Following the recommendation in "Writing R Extensions" we use exportMethods instead of export in NAMESPACE (this is described as being "clearer"). This works for head, but not for mean. Obviously we importFrom(utils, head), but don't need to do this for mean,

Hello All, I tried to intall the bigmemory package from a CRAN mirror site and received the following output while installing. Any idea what's going on and how to fix it? The system details are provided below. --------------------- begin error messages ----------------------- * installing *source* package 'bigmemory' ... checking for Sun Studio compiler...no checking for

Dear R users: Recently, I tried to write a program to calculate cross-validated predicted value. My sources are as follows. However, the R reported an error. Could you please check the sources? Thanks. set.seed(100) x<-rnorm(100) y<-sample(rep(0:1,50),replace=T) dat<-data.frame(x,y) library(rms) fito<-lrm(y~x) preo<-predict(fito) pre<-matrix(NA,nrow=100,ncol=200) for (i in

Hey, I need some help. I want to obtain a cross validation for a regression model (binary response) but I got an error with CVbinary. Well I did this: fit <- lm(resp ~ PC1 + PC2 + PC3 + PC4 + PC5 + PC6 + PC7 + PC8 + PC9+PC10+PC11+PC12+PC13+PC14+PC15+PC16+PC17+PC18+PC19+PC20+PC21+PC22+PC23+PC24+PC25+PC26+PC27+PC28, data = dexp.cp, family=binomial()) CVbinary(fit) Error in sample(nfolds, m,

Hi All,  I am trying to run cv.glmnet(x,y,family="multinomial", nfolds =4) and I only have 8 observations and the number of features I have is 1000, so my x matrix is 8 by 1000 and when I run the following, I get this error, I am not sure what is causing this problem.  Error in predmat[which, , seq(nlami)] = preds :   number of items to replace is not a multiple of replacement length Can

Dear R-help list members, I am hoping to get you help in reproducing a problem I am having That is only reproducible on a large-memory machine. Whenever I run the following lines, get a segfault listed below: *** caught segfault *** address 0x7f092cc46e40, cause 'invalid permissions' Traceback: 1: La.svd(x, nu, nv) 2: svd(X) 3: ginv(bigmatrix) Here is the code that I run:

Does anyone know a way to do the following: Save a large number of R objects to a file (like load() does) but then read back only a small named subset of them . As far as I can see, load() reads back everything. The context is: I have an application which will generate a large number of large matrices (approx 15000 matrices each of dimension 2000*30). I can generate these matrices using an

(Sorry if this is a repost, I got a bounce reply from the r-help server) Hi, I’m using the biglm() function to create some linear models for a very large data set than lm() can’t fit due to memory issues (the problem is with the number of interactions, I can fit the main effects model) I need to determine if the 2-way interactions are necessary or not. Ideally I’d like to use anova() to

Dear List, I'm running simulations using the glmnet package. I need to use an 'automated' method for model selection at each iteration of the simulation. The cv.glmnet function in the same package is handy for that purpose. However, in my simulation I have p >> N, and in some cases the selected model from cv.glmet is essentially shrinking all coefficients to zero. In this case,

Hello, good mornig, i have one questions, anybody know how to calculate svd of a matrix of the library bigmatrix with library irlba? thanks [[alternative HTML version deleted]]

I have a large file backed big. matrix, with millions of rows and 20 columns. The columns contain data that I simply need to tabulate. There are a few dozen unique values. and I just want a frequency count Test code with a small "big" matrix. library(bigmemory) library(bigtabulate) test <- big.matrix(nrow = 100, ncol = 10) test[,1:3]<- sample(150) test[,4:6]<-

Dear R users: Recently, I learn to use penalized logistic regression. Two packages (penalized and glmnet) have the function of lasso. So I write these code. However, I got different results of coef. Can someone kindly explain. # lasso using penalized library(penalized) pena.fit2<-penalized(HRLNM,penalized=~CN+NoSus,lambda1=1,model="logistic",standardize=TRUE) pena.fit2