similar to: Memory Issue while looping through CSV file

Displaying 20 results from an estimated 80 matches similar to: "Memory Issue while looping through CSV file"

2012 May 14
2
Error in names(x) <- value: 'names' attribute must be the same length as the vector
Dear R-helpers, I am stuck on an error in R: When I run my code (below), I get this error back: Error in names(x) <- value : 'names' attribute must be the same length as the vector Then when I use traceback(), R gives me back this in return: `colnames<-`(`*tmp*`, value = c(""Item", "Color" ,"Number", "Size")) I'm not exactly
2007 Jul 08
1
Writing Excel (.xls) files on non-Windows OSs using Perl
Hi all, There have been quite a few threads in the recent months pertaining to the ability to directly write native Excel (.xls) files from R. For example, exporting R matrices and/or data frames to an Excel file, with perhaps the ability to create multiple tabs (worksheets) within a single file, with one tab/sheet per R object. There exists the xlsReadWrite package on CRAN by Hans-Peter Suter,
2011 Aug 18
3
Speeding nested loops up
Hi I'm trying to speed my loop up. Any Suggestions?? At the moment it takes a few days to run. THE CODE --------------------------------------------------------------------------------------------------- for(i in 1:11) { for (j in 3:12) { for (k in 1:273107) { y[k,1] <- x[i,j,k] print(y) Rainfall_dataset <- read.table("1km_grid_nzmg.csv",
2009 Jun 23
2
syntax error, unexpected tINTEGER, expecting $end
Please help me solve the following error message: syntax error, unexpected tINTEGER, expecting $end I have this model: Code: 1. class GeoDatum < ActiveRecord::Base 2. end The database contains the table geo_data, which contains the zip_code column, filled with zip codes. I get the above error when searching for a zip code and the zip code exists in the table. Here is the
2010 Apr 14
3
PGError: duplicate key
I''m running a rails 2.3.5 / ruby 1.8.7 / postgresql 8.4.2 app and have just run into a curious problem. I have several tables that are initialized with data from csv files and I''ve noticed that, when I try to add new data to any of these tables via Table.new I get a PGError on duplicate key. For example: MIGRATION: class CreateNumbers < ActiveRecord::Migration def self.up
2007 Jun 21
2
segfault during cbind
The following code results in a seg fault. > sessionInfo() R version 2.6.0 Under development (unstable) (2007-06-21 r42013) x86_64-unknown-linux-gnu locale: LC_CTYPE=en_US;LC_NUMERIC=C;LC_TIME=en_US;LC_COLLATE=en_US;LC_MONETARY=en_US;LC_MESSAGES=en_US;LC_PAPER=en_US;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=en_US;LC_IDENTIFICATION=C attached base packages: [1] stats graphics
2007 Jun 06
2
lookup in CSV recipe
I await Luke''s node settings implementation with interest. At the moment however, I have this sort of ugliness: $site = $hostname ? { fred => "opsera", barney => "bedrock", default => "unknown site", ... } So I''ve knocked up this little function to use CSV files instead. Now I can just do: $site =
2014 Jan 27
4
Perl Search::Xapian
Hi, Trying to learn Search::Xapian and be better at perl at the same time, I'm stuck, at the DB_CREATE_OR_OPEN error. Perl says this: ~/dev/sandbox/Xapian-perl$ ./Index1-Xap.pl 100-objects-v1.csv db "db" is not exported by the Search::Xapian module Can't continue after import errors at ./Index1-Xap.pl line 7. BEGIN failed--compilation aborted at ./Index1-Xap.pl line 7. What I
2012 Jul 25
3
ff package: reading selected columns from csv
*Dear R users, Ive just started using the ff package. There is a csv file (~4Gb) with 7 columns and 6e+7 rows. I want to read only column from the file, skipping the first 100 rows. Below Ive provided different outcomes, which will clarify my problem * > sessionInfo() R version 2.14.2 (2012-02-29) Platform: x86_64-pc-mingw32/x64 (64-bit) locale: ... attached base packages: [1] tools
2012 May 09
12
out of memory (java heap space) on zip creation (jruby)
I am using rubyzip and am trying to put a huge csv file with 1.4 million rows into the zip file. Using jruby I get a out of heap error. I believe the error happens in the block below: Zip::ZipOutputStream.open(zip_path) do |zos| zos.put_next_entry(File.basename(csv_path)) zos.print IO.read(csv_path) end -- You received this message because you are subscribed to the Google
2011 Aug 15
5
Labelling all variables at once (using Hmisc label)
I have a dataset and a list of labels. I simply want to apply the labels to the variables, all at once. The only way I was able to do it was using a loop:     for (i in 1:length(data))  label(data[,i]) <- data.labels[i] I'd like to find the non-loop way to do it, using apply or the like... Any help appreciated. [[alternative HTML version deleted]]
2010 Oct 21
3
importing csv gets me all 16,000 columns with "NA"
I'm new to R. I have a mac (OS10.6). I have converted an Excel file to a csv to import into R. I have used many methods to import the file, most do not work, the best so far is: filename <- read.csv(/Users/Desktop/csvfile.csv", header=T, sep=","). I have also tried taking out the header and sep lines and it still imports fine. Regardless of what I do, it always brings in
2006 Mar 16
6
File Auditing with rails - File I/O issue
Hey Folks, I''ve written a script to generate a list of all files on a machine, then zip the results. ?I''m going to stick a web front end on it, and use it to audit some file servers. The problem is that the ZIP compression/decompression isn''t working, but its not what you might think. ?The script completes, but seems to not actually write anything. Here is a short
2012 Sep 17
1
How to filter information from a big .csv table into a new table
Hi, I have big .csv file. I would like to filter that file into a new table. For example, I have .csv file as below: f1 f2 f3 f4 f5 f6 f7 f9 f10 f11 t1 1 0 1 0 1 0 0 0 0 1 t2 1 0 0 0 0 1 1 1 1 1 t3 0 0 0 0 0 0 0 0 0 0 t4 1 0 0 0 1 0 0 0 0 0 t5 0 0 0 0 0 0 0 0 0 0 t6 0 0 0 0 0 0 0 0 0 0
2012 Sep 18
1
chunk row to new table/file
I have big .csv file. I would like to filter that file into a new table. For example, I have .csv file as below: f1 f2 f3 f4 f5 f6 f7 f9 f10 f11 t1 1 0 1 0 1 0 0 0 0 1 t2 1 0 0 0 0 1 1 1 1 1 t3 0 0 0 0 0 0 0 0 0 0 t4 1 0 0 0 1 0 0 0 0 0 t5 0 0 0 0 0 0 0 0 0 0 t6 0 0 0 0 0 0
2012 Sep 17
1
self defined distance matrix in NbClust
i m using a package NbClust for cluster analysis. in the following algorithm ->NbClust(m, diss="NULL", distance = "euclidean", min.nc=2, max.nc=15, method = "ward", index = "all", alphaBeale = 0.1) i want to define my own dissimilarity matrix of dimension 38*38. my original data "m" is a matrix of 365*38. whenever i define my own dissimilarity
2018 Apr 19
1
Syntax for capturing and writing file names
Hi all, I am looking for syntax to read a list of *txt files, (loop through?) each with a different data content but same fields, then run lines of code to produce summary stats (I have working code for this) and add the matching data file name when writing the results of a line of R code.?? Suggestions on any package vignettes that may be useful welcomed. My manual code for this is below
2012 Sep 14
1
Any way to get read.table.ffdf() (in the ff package) to pass colClasses or comment.char parameters through to read.fwf() ?
Hi everyone, my apologies if I'm overlooking something obvious in the documentation. I'm relatively inexperienced with the (awesome) ff package. My goal is to use the read.table.ffdf() function to call the read.fwf() function and pass through the colClasses and comment.char arguments. The code below shows exactly what doesn't work for me. If the colClasses and comment.char
2012 Oct 22
4
Possible to exclude an event in Puppet dashboard?
Hi All, I was wondering if it is possible to NOT show an event in puppet dashboard. I have an exec resource that fires of a ruby script. This is triggering a changed event on every run, but actually I want to exclude this from showing up as "CHANGED" Is this possible, if so how can I achieve this? --- myscript: --- exec { "run_myscript" : #
2013 Nov 29
0
Formatting output of plotKML
Hello, I previously submitted the below query to r-sig-geo, but have had no response. Before I start bothering individual maintainers, I wonder if anyone on this list has any experience with the package and (or!) can diagnose my problems? Thanks, Calum Hello, I am having a little trouble with plotKML and I am not sure whether to ascribe it to my incompetence (most likely), incomplete