R help - Aug 2008 - Combining multiple datasets

Hi,
I've tried to figure this out using Intro to R and help(), to no avail 
- I am new at this.

I'm trying to write a script that will read multiple files from a 
directory and then merge them into a single new data frame.
The original data are in a tree-ring specific format, and so I've first 
used a function (read.rwl) from the dplR package to read each file, 
translate each into a more intuitive time series format, and then put 
each new data frame into a single object, demo.Rwl:

 >demo.Rwl <- list.files(pattern=".RWL$"); for(i in demo.Rwl) {
x <-
read.rwl(i, header=TRUE); assign(print(i, quote=FALSE), x)}

This part seems to work.  Now, I want to be able to put all of the 
individual data frames contained in demo.Rwl into a single data frame, 
merging by rows (rows are calendar years in the time series). I think I 
know how to do this by typing each data set name into a merge:

 >merge(x,y..., by.x=0, all=TRUE )

However, I'd like to be able to script something that will save me the 
time of typing in all of the individual data set names, as I will be 
repeating this operation many times with many different numbers of 
original input files. Is there a way to code a merge such that every 
individual data frame contained in demo.Rwl (a different number every 
time) gets combined into a single data set?

Thanks for the help!

Ali

Here is the way that I usually do it using lapply:

myData <- lapply(list.files(pattern=".RWL$"), function(.file)
read.rel(.file, header=TRUE))
# combine into a single dataframe
myDF <- do.call(rbind, myData)



On Thu, Aug 21, 2008 at 10:42 PM, Alison Macalady <ali at kmhome.org>
wrote:
> Hi,
> I've tried to figure this out using Intro to R and help(), to no avail
- I
> am new at this.
>
> I'm trying to write a script that will read multiple files from a
directory
> and then merge them into a single new data frame.
> The original data are in a tree-ring specific format, and so I've first
used
> a function (read.rwl) from the dplR package to read each file, translate
> each into a more intuitive time series format, and then put each new data
> frame into a single object, demo.Rwl:
>
>>demo.Rwl <- list.files(pattern=".RWL$"); for(i in demo.Rwl)
{ x <-
>> read.rwl(i, header=TRUE); assign(print(i, quote=FALSE), x)}
>
> This part seems to work.  Now, I want to be able to put all of the
> individual data frames contained in demo.Rwl into a single data frame,
> merging by rows (rows are calendar years in the time series). I think I
know
> how to do this by typing each data set name into a merge:
>
>>merge(x,y..., by.x=0, all=TRUE )
>
> However, I'd like to be able to script something that will save me the
time
> of typing in all of the individual data set names, as I will be repeating
> this operation many times with many different numbers of original input
> files. Is there a way to code a merge such that every individual data frame
> contained in demo.Rwl (a different number every time) gets combined into a
> single data set?
>
> Thanks for the help!
>
> Ali
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>


-- 
Jim Holtman
Cincinnati, OH
+1 513 646 9390

What is the problem that you are trying to solve?

Here's a function that does what you asked, you may need to adjust the
column
names of your data frames before using it. If all your data frames are
similar (same number of rows, same years) then try do.call('cbind',
yourList).


#This function takes a list of data frames and merges them into one data
frame.
#	The data frames are assumed to have one common column name (the one used 
#	for joins) and customized name(s) for the remaining column(s).
#LDF = list of data frames to be merged

rmerge <- function(LDF, verbose=FALSE)
{
DF <- LDF[[1]]
cat(paste("Started with", nrow(DF), "rows from",
names(LDF)[1]), "\n")
for (i in 2:length(LDF)) {
	DF <- merge(DF, LDF[[i]], all=TRUE)  #outer join
	if (verbose) cat(paste("Adding ", nrow(LDF[[i]]), " rows from
",
names(LDF)[i], 
		" (", nrow(DF), " rows so far)...", sep=""),
"\n")
}
rownames(DF) <- NULL
cat(paste("Got", nrow(DF), "rows after recursive merging"),
"\n")
DF
}




Alison Macalady wrote:
> 
> Hi,
> I've tried to figure this out using Intro to R and help(), to no avail 
> - I am new at this.
> 
> I'm trying to write a script that will read multiple files from a 
> directory and then merge them into a single new data frame.
> The original data are in a tree-ring specific format, and so I've first
> used a function (read.rwl) from the dplR package to read each file, 
> translate each into a more intuitive time series format, and then put 
> each new data frame into a single object, demo.Rwl:
> 
>  >demo.Rwl <- list.files(pattern=".RWL$"); for(i in
demo.Rwl) { x <-
> read.rwl(i, header=TRUE); assign(print(i, quote=FALSE), x)}
> 
> This part seems to work.  Now, I want to be able to put all of the 
> individual data frames contained in demo.Rwl into a single data frame, 
> merging by rows (rows are calendar years in the time series). I think I 
> know how to do this by typing each data set name into a merge:
> 
>  >merge(x,y..., by.x=0, all=TRUE )
> 
> However, I'd like to be able to script something that will save me the 
> time of typing in all of the individual data set names, as I will be 
> repeating this operation many times with many different numbers of 
> original input files. Is there a way to code a merge such that every 
> individual data frame contained in demo.Rwl (a different number every 
> time) gets combined into a single data set?
> 
> Thanks for the help!
> 
> Ali
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
> 
> 
-- 
View this message in context:
http://www.nabble.com/Combining-multiple-datasets-tp19100369p19108774.html
Sent from the R help mailing list archive at Nabble.com.