R help - Oct 2010 - Import Multiple csv files and merge into one Master file

Dear R Group:

    How to import multiple csv files and merge into one dataset.

Thanks and Regards,

   Xing

	[[alternative HTML version deleted]]

Hi Xing,

This depends somewhat on what you mean by "merge", and how many files
you are talking about.  Supposing you are dealing with few enough
files you can do it manually:

dat1 <- read.csv("yourfile1.csv")
dat2 <- read.csv("yourfile2.csv")
...
datn <- read.csv("yourfilen.csv")

If each file contains unique variables:

complete.dat <- cbind(dat1, dat2, ... , datn)

if each one is just a continuation rowise:

complete.dat <- rbind(dat1, dat2, ... , datn)

If they are all sort of related but in no consistent way, and do not need to be:

complete.dat <- list(dat1, dat2, ... , datn)

If you need some fancier merging than just columnwise or rowwise, look
at merge().  For documentation on these features see:

?data.frame # to find out more about data frames (which is what read.csv uses)
?list # for details about what a list is
?read.table
?read.csv # just a special wrapper for read.table
?cbind # for column binding
?rbind # for row binding
?merge # for more specialized merging
example(merge) # for examples using merge()

HTH,

Josh

On Thu, Oct 7, 2010 at 8:19 PM, XINLI LI <lihawaii at gmail.com>
wrote:
> Dear R Group:
>
> ? ?How to import multiple csv files and merge into one dataset.
>
> Thanks and Regards,
>
> ? Xing
>
> ? ? ? ?[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>


-- 
Joshua Wiley
Ph.D. Student, Health Psychology
University of California, Los Angeles
http://www.joshuawiley.com/

See the R Data Import/Export manual for the first step:

http://cran.r-project.org/doc/manuals/R-data.html

?read.table should help you out.  You might use
?lapply along with read.table to read in multiple files.

Then, use ?merge, possibly in tandem with the ?Reduce
function, depending on how many data.frames you're
dealing with.

A more specific question will elicit perhaps more specific
answers.


On 10/07/2010 10:19 PM, XINLI LI wrote:
> Dear R Group:
>
>      How to import multiple csv files and merge into one dataset.
>
> Thanks and Regards,
>
>     Xing
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

Hi Xinli,

You will probable have to tweak this some for it to work for you, but
it at least gives you an idea.

First put all your files in one directory, then you use the
list.files() function to read into R the names of every file in that
directory (this is easier than typing all 100 something names).  Now,
you can use lapply() to 'apply' the function, read.csv() to each file
name.  In my example, I set header = TRUE just to show you how you can
specify arguments to the function you are calling with lapply().  This
will result in a list with each element being the results of
read.csv().  Finally, since all the columns are the same, we can just
rbind() every data frame together.  That is accomplished the outer
function, do.call(), whose first argument is "rbind" and second is the
output of lapply().

filenames <- list.files(path = "~/")
do.call("rbind", lapply(filenames, read.csv, header = TRUE))

Hope that helps,

Josh

On Fri, Oct 8, 2010 at 4:32 AM, XINLI LI <lihawaii at gmail.com>
wrote:
> Hi Joshua:
>
> ??? Thank you very much for your help. I have more than 100 files, and do
> you have a better way to merge the files into one dataset, there all have
> the same columns.
>
> ?? Thanks,
>
> ?? xinli
>
> On Thu, Oct 7, 2010 at 11:28 PM, Joshua Wiley <jwiley.psych at
gmail.com>
> wrote:
>>
>> Hi Xing,
>>
>> This depends somewhat on what you mean by "merge", and how
many files
>> you are talking about. ?Supposing you are dealing with few enough
>> files you can do it manually:
>>
>> dat1 <- read.csv("yourfile1.csv")
>> dat2 <- read.csv("yourfile2.csv")
>> ...
>> datn <- read.csv("yourfilen.csv")
>>
>> If each file contains unique variables:
>>
>> complete.dat <- cbind(dat1, dat2, ... , datn)
>>
>> if each one is just a continuation rowise:
>>
>> complete.dat <- rbind(dat1, dat2, ... , datn)
>>
>> If they are all sort of related but in no consistent way, and do not
need
>> to be:
>>
>> complete.dat <- list(dat1, dat2, ... , datn)
>>
>> If you need some fancier merging than just columnwise or rowwise, look
>> at merge(). ?For documentation on these features see:
>>
>> ?data.frame # to find out more about data frames (which is what
read.csv
>> uses)
>> ?list # for details about what a list is
>> ?read.table
>> ?read.csv # just a special wrapper for read.table
>> ?cbind # for column binding
>> ?rbind # for row binding
>> ?merge # for more specialized merging
>> example(merge) # for examples using merge()
>>
>> HTH,
>>
>> Josh
>>
>> On Thu, Oct 7, 2010 at 8:19 PM, XINLI LI <lihawaii at gmail.com>
wrote:
>> > Dear R Group:
>> >
>> > ? ?How to import multiple csv files and merge into one dataset.
>> >
>> > Thanks and Regards,
>> >
>> > ? Xing
>> >
>> > ? ? ? ?[[alternative HTML version deleted]]
>> >
>> > ______________________________________________
>> > R-help at r-project.org mailing list
>> > https://stat.ethz.ch/mailman/listinfo/r-help
>> > PLEASE do read the posting guide
>> > http://www.R-project.org/posting-guide.html
>> > and provide commented, minimal, self-contained, reproducible code.
>> >
>>
>>
>>
>> --
>> Joshua Wiley
>> Ph.D. Student, Health Psychology
>> University of California, Los Angeles
>> http://www.joshuawiley.com/
>
>


-- 
Joshua Wiley
Ph.D. Student, Health Psychology
University of California, Los Angeles
http://www.joshuawiley.com/