That seems right to me.
You might want to look into a persistent connection as in Example 10 at
https://code.google.com/p/sqldf/#Example_10._Persistent_Connections
This will save reloading the DB for every query.
You might even want to create a permanent DB as in Example 9.
This is one of my favorite packages - kudos to Gabor again!
-- David
-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org]
On Behalf Of Doran, Harold
Sent: Thursday, September 18, 2014 10:38 AM
To: r-help at r-project.org
Subject: [R] Using read.csv.sql() to read in specific columns
I am dealing with data frames that have thousands of columns and hundreds of
thousands of rows and only need a few specific columns from the data. The data
take various formats, but normally are tab-delimited.
I have written the following which is working as expected. However, because I?m
so new at using sqldf(), just looking for some verification from users that this
is in fact efficient and correct in the R-ish sense of the word and
generalizable to larger data sets.
Harold
tmp <- data.frame(replicate(50, rnorm(10)))
names(tmp) <- paste('item', 1:50, sep='')
write.table(tmp, 'tmp.txt')
read.csv.sql("tmp.txt", sql = "select item1, item2, item50 from
file", sep = ' ')
[[alternative HTML version deleted]]
This e-mail and any materials attached hereto, including, without limitation,
all content hereof and thereof (collectively, "XR Content") are
confidential and proprietary to XR Trading, LLC ("XR") and/or its
affiliates, and are protected by intellectual property laws. Without the prior
written consent of XR, the XR Content may not (i) be disclosed to any third
party or (ii) be reproduced or otherwise used by anyone other than current
employees of XR or its affiliates, on behalf of XR or its affiliates.
THE XR CONTENT IS PROVIDED AS IS, WITHOUT REPRESENTATIONS OR WARRANTIES OF ANY
KIND. TO THE MAXIMUM EXTENT PERMISSIBLE UNDER APPLICABLE LAW, XR HEREBY
DISCLAIMS ANY AND ALL WARRANTIES, EXPRESS AND IMPLIED, RELATING TO THE XR
CONTENT, AND NEITHER XR NOR ANY OF ITS AFFILIATES SHALL IN ANY EVENT BE LIABLE
FOR ANY DAMAGES OF ANY NATURE WHATSOEVER, INCLUDING, BUT NOT LIMITED TO, DIRECT,
INDIRECT, CONSEQUENTIAL, SPECIAL AND PUNITIVE DAMAGES, LOSS OF PROFITS AND
TRADING LOSSES, RESULTING FROM ANY PERSON?S USE OR RELIANCE UPON, OR INABILITY
TO USE, ANY XR CONTENT, EVEN IF XR IS ADVISED OF THE POSSIBILITY OF SUCH DAMAGES
OR IF SUCH DAMAGES WERE FORESEEABLE.