similar to: read.csv trap

I'm sure this has gotten some attention before, but I have two CSV files generated from vmstat and free that are roughly 6-8 Mb (about 80,000 lines) each. When I try to use read.csv(), R allocates all available memory (about 4.9 Gb) when loading the files, which is over 300 times the size of the raw data. Here are the scripts used to generate the CSV files as well as the R code: Scripts (run

Hi, I have a script below. dat <- read.table(file="pt.csv", header=T, sep=",", row.names=1, col.names=1) dat for(which_col in seq_len(ncol(dat))) { subset_data <- dat[,which_col:ncol(dat)] file_name <- sprintf('%s.csv', colnames(dat)[which_col]) write.csv(subset_data, file_name) message(sprintf('Saving %s', file_name)) }

Consider: > x <- matrix(1:6, 2,3) > rownames(x) <- c("ID1", "ID2") > colnames(x) <- c("Attr1", "Attr2", "Attr3") > x Attr1 Attr2 Attr3 ID1 1 3 5 ID2 2 4 6 > write.csv(x,file="x.csv") "","Attr1","Attr2","Attr3" "ID1",1,3,5

Dear R experts, I am trying to import some data from some Excle files into R. My Excle file contains about 50 sheets. One solution I can think about is to convert my Excle file into csv file first and then load data into R using 'read.csv'. But it seems to me that 'read.csv' only supports reading one sheet (or 'one file') each time, so that seems I have to create 50 csv

Dear R helpers, Following is my R code where I am trying to calculate returns and then trying to create a data.frame. Since, I am not aware how many instruments I will be dealing so I have constructed a function. My R code is as follows - library(plyr) mydata <- data.frame(instru_name =

Hi all, I have some trouble reading data from a csv file. I used command "read.delim("clipboard") to read in the data. > aalpha.data <- read.delim("clipboard") > class(aalpha.data) [1] "data.frame" > dim(aalpha.data) [1] 8 25 > colnames(aalpha.data) [1] "X" "V1" "V2" "V3" "V4"

Hello, I'm looking for a solution for the following problem: 1) I have a folder with several csv files; each contains a set of measurement values 2) The measurements of each file belong to a position in a two dimensional matrix (lets say "B02.csv" belongs to position 2,2 3) The size of the matrix is fix 4) I cannot assure to have a csv file for each position 5) Each position

I had occasion recently to read in a one-line *.csv file that looked like: "CandidateName","NSN","Ethnicity","dob","gender" "Smith, Mary Jane",111222333,"E","2/25/1989","F" That "F" (for female) in the last field got transformed to FALSE. Apparently read.csv (and hence read.table) are inferring

I'm working with a 350MB CSV file on a server that has 3GB of RAM, yet I'm hitting a memory error when I try to store the data frame into a survey design object, the R object that stores data for complex sample survey data. When I launch R, I execute the following line from Windows: "C:\Program Files\R\R-2.9.1\bin\Rgui.exe" --max-mem-size=2047M Anything higher, and I get an

Hi, I have a table which I can read via read.csv: fx1<-read.csv(file="A_FX_M.csv", header=TRUE) TIME REER NTD JPY GBP HKD 1 198001 124.26 36.030 237.96 2.263980 4.8366 2 198002 126.59 36.030 244.05 2.290426 4.8765 3 198003 128.33 36.026 248.62 2.206045 4.9960 4 198004 127.85 36.063 251.67 2.215330 4.9760 5 198005 124.40 36.050 228.35 2.302026 4.8891 6 198006

Dear R users, Suppose the csv file contains header names such as *"Nike, dunk"*, *"Converse, All stars"* etc When imported to R (with header = T option), the column names are given by: *"Nike..dunk"* *"Converse..All.stars"* I have tried the following command to convert these column names to the original names as in the input source.

Hi!! I am trying to generate data with specific correlation using corgen method from the ecodist package. It does not seem to generate the data properly. I am listing the code below. Secondly, when I put the code given below all in one file and source it, it is unable to complete the task. But when I execute it one at a time it is able to complete it. Can someone point out any mistake that I may

Dear R Forum, I have a data.frame as df = data.frame(date = c("2013-04-15", "2013-04-14", "2013-04-13", "2013-04-12", "2013-04-11"), ABC_f = c(62.80739769,81.04525895,84.65712455,12.78237251,57.61345256), LMN_d = c(21.16794336,54.6580401,63.8923307,87.59880367,87.07693716), XYZ_p = c(55.8885464,94.1358684,84.0089114,98.99746696,64.71083712),

Hello list, Thanks in advance for any help. I have many (approx 20) files that I have merged. For example d1<-read.csv("AlleleReport.csv") d2<-read.csv("AlleleReport.csv") m1 <- merge(d1, d2, by = c("IND", intersect(colnames(d1), colnames(d2))), all = TRUE) m2 <- merge(m1, d3, by = c("IND", intersect(colnames(m1), colnames(d3))), all =

Dear R users, I have 14 csv's files that I want read in R, but one of them I can't read (the others 13 don't bring me any problem), and I don't know the problem. I will report the subject of the csv that I can't read, one csv that don't have any problem to read and the script that I use to try do it. CSV THAT I CAN'T READ: *****************************

Dear R helpers, I have one trivial problem while writing an output file in csv format. I have two dataframes say df1 and df2 which I am reading from two different csv files. df1 has column names as date, r1, r2, r3 while the dataframe df2 has column names as date, 1w, 2w. (the dates in both the date frames are identical also no of elements in each column are equal say = 10). I merge these

Hi, I'm a R newbie and want to accelerate the following pre-filtering step of a data set with more than 115,000 rows : #----------------- # Function to filter out constant data columns filter.const<-function(X, vectors=c('column', 'row'), tol=0){ realdata=c() filteredX<-matrix() if( vectors[1] == 'row' ){ for( row in (1:nrow(X)) ){ if(

Full_Name: Ed Borasky Version: 1.8.1 OS: Windows XP Professional Submission from: (NULL) (208.252.96.195) R 1.8.1 seems to be running into a memory allocation problem in the "aggregate" function. I have a rather large dataset (14 columns by 223,000 rows -- almost 40 megabytes) and a script that performs some processing on it. The system is a 768 MB Pentium 4. Here's the console

HI, I have a file named "s1"(which has 14 rows and 4 columns) which is in csv format. I import and call it "A" the file using: A <- read.csv(file="s1.csv",head=TRUE,sep=",") then I transpose it and call it "B"(which will have 4 rows and 14 columns) using: B=t(A) when I want to make 5 samples from "B" (choosing 10 columns) using:

bob<-read.csv('shi.csv', header=T) newmean<-matrix(0, test, dim(bob)[2]-6);a<-0; for (i in c(4,8:(dim(bob)[2]))) {a<-a+1;newmean[,a]<-tapply(bob[,i], bob$Exam, mean)} colnames(newmean)<-colnames(bob)[c(4,8:(dim(bob)[2]))] Could anyone please help me what does the above code does ... I want to find mean ... but would like to know what exactly is the above code doing.