Hi, I was trying to read & filter records from a large file using read.csv.sql. I was successfully able to do that with a ~1 GB file. However I get the following error with a >2 GB file which has exactly the same structure as the first file. *Error in try({ : * * RS-DBI driver: (RS_sqlite_import: cannot open file <filename> )* -I have checked the file permission and the file exists in the directory. -The error comes instantly, so I am assuming its not due to memory. -If I skip significant number of rows(using skip= statement) from the > 2GB file, I don't get the error. I can do a quick and dirty fix by splitting the file into two and by performing two separate reads, but may be thats not so elegant. Can somebody please help? -- [[alternative HTML version deleted]]
Gabor Grothendieck
2010-Apr-06 10:57 UTC
[R] Error with read.csv.sql on processing large file
1. Are you able to determine if its the filtering step that is the problem? Also is it the size of the file or is it a particular record that is causing the problem? If you read it in in two pieces do both pieces work or just one piece? It might be a problem in the filtering step (in which case you could try filtering it yourself outside of R and then try reading that in) or there might be something wrong with one of the records in which case you need to narrow down which one by reading in portions. 2. Can you provide more info? Platform? Exactly what command you are using? 3. In some cases the problem was the one in FAQ #11 http://code.google.com/p/sqldf/#11._Why_am_I_having_difficulty_reading_a_data_file_using_SQLite On Tue, Apr 6, 2010 at 1:57 AM, Moiz Saifee <moiz.saifee at gmail.com> wrote:> Hi, > > I was trying to read & filter records from a large file using read.csv.sql. > I was successfully able to do that with a ~1 GB file. However I get the > following error with a >2 GB file which has exactly the same structure as > the first file. > > *Error in try({ : * > * ?RS-DBI driver: (RS_sqlite_import: cannot open file <filename> )* > > -I have checked the file permission and the file exists in the directory. > -The error comes instantly, so I am assuming its not due to memory. > -If I skip significant number of rows(using skip= statement) from the > 2GB > file, I don't get the error. > > I can do a quick and dirty fix by splitting the file into two and by > performing two separate reads, but may be thats not so elegant. Can somebody > please help? > > -- > > ? ? ? ?[[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >