R help - Aug 2009 - list indexing problem reading in files from a directory

I am running into a problem with allocating these files to a list as
they are read in through a for loop.  I know I am probably doing
something wrong, but I can't figure out what.  I know this is not
reproducible.  I am ending up with a data frame of the very last file
to be read in.  I know it is in the indexing, but I can't wrap my head
around it.
thanks for all of the help,

Stephen Sefick



#level logger read in
read.ll <- function(path){
library(chron)
library(zoo)
list.of.files <- list.files(path)
length.files <- length(list.of.files)
	for(i in 1:length.files){
	df <- list()
	df[[i]] <- read.table(paste(sep="/", path, list.of.files[i]),
skip=45)
	length.1 <- length(df[[i]][,1])
	length.2 <- length(df[[i]][,1])-1
	df[[i]] <- df[[i]][-c(length.1, length.2),]
	df[[i]] <- data.frame(chron(as.character(df[[i]][,1]),
as.character(df[[i]][,2]), format=c(dates="Y/m/d",
times="H:M:S")),
as.numeric(df[[i]][,3]), as.numeric(df[[i]][,4]),
as.factor(list.of.files[[i]]))

	}
}

a <- read.ll("C:/Documents and Settings/Feminella
Lab/Desktop/WB_LL_Data/Wolf Bay Data/Raw solinist data/Compensated by
Brad")

-- 
Stephen Sefick

Let's not spend our time and resources thinking about things that are
so little or so large that all they really do for us is puff us up and
make us feel like gods.  We are mammals, and have not exhausted the
annoying little problems of being mammals.

								-K. Mullis

On Tue, 11 Aug 2009, stephen sefick wrote:
> I am running into a problem with allocating these files to a list as
> they are read in through a for loop.  I know I am probably doing
> something wrong, but I can't figure out what.  I know this is not
> reproducible.  I am ending up with a data frame of the very last file
> to be read in.  I know it is in the indexing, but I can't wrap my head
> around it.
> thanks for all of the help,
>
> Stephen Sefick
>
>
>
> #level logger read in
> read.ll <- function(path){
> library(chron)
> library(zoo)
> list.of.files <- list.files(path)
> length.files <- length(list.of.files)
> 	for(i in 1:length.files){
> 	df <- list()
move the above line to BEFORE the loop and you should be good to go.

HTH,

Chuck
> 	df[[i]] <- read.table(paste(sep="/", path, list.of.files[i]),
skip=45)
> 	length.1 <- length(df[[i]][,1])
> 	length.2 <- length(df[[i]][,1])-1
> 	df[[i]] <- df[[i]][-c(length.1, length.2),]
> 	df[[i]] <- data.frame(chron(as.character(df[[i]][,1]),
> as.character(df[[i]][,2]), format=c(dates="Y/m/d",
times="H:M:S")),
> as.numeric(df[[i]][,3]), as.numeric(df[[i]][,4]),
> as.factor(list.of.files[[i]]))
>
> 	}
> }
>
> a <- read.ll("C:/Documents and Settings/Feminella
> Lab/Desktop/WB_LL_Data/Wolf Bay Data/Raw solinist data/Compensated by
> Brad")
>
> -- 
> Stephen Sefick
>
> Let's not spend our time and resources thinking about things that are
> so little or so large that all they really do for us is puff us up and
> make us feel like gods.  We are mammals, and have not exhausted the
> annoying little problems of being mammals.
>
> 								-K. Mullis
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
Charles C. Berry                            (858) 534-2098
                                             Dept of Family/Preventive Medicine
E mailto:cberry at tajo.ucsd.edu	            UC San Diego
http://famprevmed.ucsd.edu/faculty/cberry/  La Jolla, San Diego 92093-0901

On Tue, 2009-08-11 at 12:45 -0500, stephen sefick wrote:
> I am running into a problem with allocating these files to a list as
> they are read in through a for loop.  I know I am probably doing
> something wrong, but I can't figure out what.  I know this is not
> reproducible.  I am ending up with a data frame of the very last file
> to be read in.  I know it is in the indexing, but I can't wrap my head
> around it.
> thanks for all of the help,
> 
> Stephen Sefick
A missing return() or explicit statement about what should be return
from your function.

Your function is not right either. You keep killing df at the start of
each iteration of the for loop so df is a list with a single component
each time.
> 
> 
> #level logger read in
> read.ll <- function(path){
> library(chron)
> library(zoo)
> list.of.files <- list.files(path)
> length.files <- length(list.of.files)
> 	for(i in 1:length.files){
> 	df <- list()
Move the line above outside of the loop, above. Better would be to
allocate storage so you know everything is correct, rather than an empty
list. It doesn't improve speed at all IIRC, but is a bit tidier, so try

df <- vector(mode = "list", length = length.files)

immediately after length.files <- ....
> 	df[[i]] <- read.table(paste(sep="/", path, list.of.files[i]),
skip=45)
> 	length.1 <- length(df[[i]][,1])
> 	length.2 <- length(df[[i]][,1])-1
> 	df[[i]] <- df[[i]][-c(length.1, length.2),]
> 	df[[i]] <- data.frame(chron(as.character(df[[i]][,1]),
> as.character(df[[i]][,2]), format=c(dates="Y/m/d",
times="H:M:S")),
> as.numeric(df[[i]][,3]), as.numeric(df[[i]][,4]),
> as.factor(list.of.files[[i]]))
> 
> 	}
Here you probably want

return(df)

or just

df

But I prefer return(....) as it is pretty explicit about your intentions
that, in this case, 'df' is what you want the function to return.

You function is probably return the last value of the for loop, but as I
say, your loop can't possibly work as you keep killing the list so it
will only ever contain a single processed data.frame.

You could check this yourself by debugging, by:

debug(read.ll)

Then call your function and step through, and at each stage in the loop,
see what df contains. If you do this on the function as you posted,
you'll quickly see what the problem is.

HTH

G
> }
> 
> a <- read.ll("C:/Documents and Settings/Feminella
> Lab/Desktop/WB_LL_Data/Wolf Bay Data/Raw solinist data/Compensated by
> Brad")
> 
-- 
%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%
 Dr. Gavin Simpson             [t] +44 (0)20 7679 0522
 ECRC, UCL Geography,          [f] +44 (0)20 7679 0565
 Pearson Building,             [e] gavin.simpsonATNOSPAMucl.ac.uk
 Gower Street, London          [w] http://www.ucl.ac.uk/~ucfagls/
 UK. WC1E 6BT.                 [w] http://www.freshwaters.org.uk
%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%