similar to: memory usage grows too fast

Displaying 20 results from an estimated 300 matches similar to: "memory usage grows too fast"

2009 Sep 04
2
lrm in Design package--missing value where TRUE/FALSE needed
Hi, A error message arose while I was trying to fit a ordinal model with lrm() I am using R 2.8 with Design package. Here is a small set of mydata: RC RS Sex CovA CovB CovC CovD CovE 2 1 0 1 1 0 -0.005575280 2 2 1 0 1 0 1 -0.001959580 2 3 0 0 0 1 0 -0.004725880 2 0 0 0 1 0 0 -0.005504850 2 2 1 1 0 0 0 -0.003880170 1 2 1 0 0 1 0 -0.006074230 2 2 1 0 0 1 1 -0.003963920 2 2 1 0 0 1 0
2009 Jul 02
1
superimpose multiple scatterplot matrices
Dear R- experts, I am trying to superimpose two or more scatterplot matrices generated by pairs() to visualize the differences over each datasets, but have not been very successful. Two data frames, df1 and df2 for example, each has the same five variables in columns. My goal is to paint each dataset with a color on the same plot panel for each pair of the five variables. That is, in this
2009 Sep 23
1
read.delim very slow in reading files with lots of columns
Hi, I am trying to read a tab-delimited file into R (Ver. 2.8). The machine I am using is 64bit Linux with 16 GB. The file is basically a matrix(~600x700000) and as large as 3GB. The read.delim() ran extremely slow (hours) even with a subset of the file (31 MB with 6x700000) I monitored the memory usage, and found it constantly only took less than 1% of 16GB memory. Does read.delim()
2010 Sep 08
3
Regression using mapply?
Hi, I have huge matrices in which the response variable is in the first column and the regressors are in the other columns. What I wanted to do now is something like this: #this is just to get an example-matrix DataMatrix <- rep(1,1000); Disturbance <- rnorm(900); DataMatrix[101:1000] <- DataMatrix[101:1000]+Disturbance; DataMatrix <- matrix(DataMatrix,ncol=10,nrow=100); #estimate
2011 Feb 10
3
Permuting rows of a matrix
Hi, I need to permute the rows of a matrix, where each row is independently rearranged. A simple solution is this: shuffled <- datamatrix <- matrix(1:24, ncol = 4) for (i in 1:nrow(datamatrix)) { shuffled[i, ] <- sample(datamatrix[i, ]) } > datamatrix [,1] [,2] [,3] [,4] [1,] 1 7 13 19 [2,] 2 8 14 20 [3,] 3 9 15 21 [4,] 4 10 16 22 [5,]
2008 Nov 18
2
sequencially merge multiple files in a folder
Dear all, If the question is too easy, please forgive me since I am only few weeks old in R. I have worked on this question a few days and still cannot figure it out. Here I have a folder with more than 50 tab-delimited files. Each file has a few hundreds of thousands rows/subjects, and the number of columns/variables of each file varies.The 1st row consists of all the variable names. Now I
2010 Dec 07
2
longer object length is not a multiple of shorter object length
In datamatrix[, "y"] == datamatrix[, "y"][-1] : longer object length is not a multiple of shorter object length out = c(FALSE,datamatrix[,'y'] == datamatrix[,'y'][-1]) and I do not know why I get that error, the resulting out matrix is somehow one row larger than datamatrix... all I try to do is filter matrix by dropping rows where [,'y'][-1] ==
2009 Sep 28
1
plot error -- figure margins too large
Hi, I am trying to plot my dataset, consisting of one column with numeric values and one column with group IDs. The set is similar to the following df. df <- NULL for ( i in 1:20) { tmp1 <- runif(1000,0,5) tmp2 <- cbind(tmp1,i) df <- rbind(df,tmp2) } Now I would like to plot the numeric column, stratified by the group IDs, in a single figure. First, I partitioned the frame
2007 Feb 13
1
Questions about results from PCAproj for robust principal component analysis
Hi. I have been looking at the PCAproj function in package pcaPP (R 2.4.1) for robust principal components, and I'm trying to interpret the results. I started with a data matrix of dimensions RxC (R is the number of rows / observations, C the number of columns / variables). PCAproj returns a list of class princomp, similar to the output of the function princomp. In a case where I can
2004 Jul 13
5
Help with factanal and missing values
Hi list, I'm performing a series of confirmatory factor analysis on different groupings of items from data collected with questionnaires. There are some missing values. For those sets with no missing values I call factanal(datamatrix,factors=n) where datamatrix is a table of all observations for the items under investigation. This call fails when there are missing values. help(factanal)
2008 Mar 03
2
handling big data set in R
Hello R users, I'm wondering whether it is possible to manage big data set in R? I have a data set with 3 million rows and 3 columns (X,Y,Z), where X is the group id. For each X, I need to run 2 regression on the submatrix. I used the function "split": datamatrix<-read.csv("datas.csv", header=F, sep=",") dim(datamatrix) # [1] 2980523 3
2010 Aug 22
4
how to implement string pattern extraction in R
Hi, In perl, to get a substring matching a particular pattern can be implemented like the following example: $x = "AAAA.txt"; if ($x=~ /(.*?)\.txt/){ $prefix = $1; } So how to do the same thing in R? Can someone provide me the code sample? Thanks much in advance. -- Waverley @ Palo Alto
2012 Jul 24
4
ERROR : cannot allocate vector of size (in MB & GB)
Hi, Here in R, I need to load a huge file(.csv) , its size is 200MB. [may come more than 1GB sometimes]. When i tried to load into a variable it taking too much of time and after that when i do cbind by groups, getting an error like this " Error: cannot allocate vector of size 82.4 Mb " My requirement is, spilt data from Huge-size-file(.csv) to no. of small csv files. Here i will give
2004 Apr 22
3
fill up a matrix
Hi all, I want to fill in this matrix vectors (by column) without overwriting the first elements in column 1995. Is there any other way than concatenate the first element with the vector and then assign this new vector to the column in matrix? matrix[,"3"]<-c(1591,"vector") matrix[,"4"]<-c(405,"vector") ... ...
2011 Nov 21
5
R ignores number only with a nine under 10000
Hello R users, I'm trying to replace numerical values in a datamatrix with strings. R does this except for numbers under 10000 starting with a 9 (eg 98, 970, 9504 etc). This is really weird and I wondered whether someone had encountered such a problem or knows the solution. I'm using the next script: test_1 <- read.table("5+ref_151111clusters3.csv", header = TRUE, sep =
2010 Nov 20
2
Xen Live Migration Fail
Dear all: I am trying to migrate a WinXP HVM (64M memory) from a machine (sender) to another machine (receiver) through AoE disk, however, there are some errors when do the following command: (My xen version is xen-4.0.1-rc5.hg, kernel 2.6.31.13, xen xen-4.0.1-rc6-pre, Both sender and receiver are the same xen version) xm migrate --live domain_id receiver_ip
2004 Apr 27
1
beginners k means clustering question
Hi all, I am wandering.. is it possible to cluster data which is in a single column ? for example.. I have some data as follows: 4013 7362 7585 9304 11879 14785 21795 30500 30669 30924 33988 36975 40422 42911 50501 51593 53729 54338 55497 57337 61993 62601 66229 69815 69933 70760 71340 75921 83972 90134 91061 . . . is it possible to cluster this data since it is in a single column ? I have
2009 Jan 20
1
heatmap.2 color issue
Dear All: I tried to use heatmap.2 to generate hierarchical clustering using the following command: heatmap.2(datamatrix, scale="row", trace="none", col=greenred(256), labRow=genelist[,1], margins=c(10,10), Rowv=TRUE, Colv=TRUE) datamatrix is subset of a RMA normalized data subset by a genelist. The problem is a lot of times, the z-score in key are from, like -5 to 15 or
2005 Jan 21
6
Avoiding a Loop?
Dear R-Helpers, I have a matrix where the first column is known. The second column is the result of multiplying this first column with a constant "const". The third column is the result of multiplying the second column with "const"..... So far, I did it like this (as a simplified example): nr.of.columns <- 4 myconstant <- 27.5 mymatrix <- matrix(numeric(0), nrow=5,
2012 Aug 10
1
Split CSV as per file size
Hi here i have a code to split a csv file as per group of line. The code given below, ------------------------------------ SplitCSVByLine <- function(DataMatrix,Destination,NoOfLineToGroup) { input <- file(DataMatrix, "r") fileNo <- 1 repeat { myLines <- readLines(input, n=NoOfLineToGroup) if (length(myLines) == 0) break