Debs Majumdar
2012-Jun-28 05:49 UTC
[R] Storing results in a single file after looping over all files
Hi All,
I have a whole lot of *.raw files in my working folder and I am doing the same
analysis on each of those and want to save all the results in a single file. I
am making some mistake here and can't figure out how to solve it.
Say, the *.raw files are ABCD.raw, EFGH.raw, IJKL.raw ...
The files are of this format
ID PHI?? aa1? aa2? aa3 ....
1??? 1???? 1.3?? 2.0?? 1.0
2??? 0???? 1.5?? NA?? 0.9
3??? 1???? 0.1?? 0.2?? 1.5
......
..
My code is as follows:
files <- list.files(pattern="*.raw")
for(i in files){
of <- strsplit(i, "\\.")[[1]]
of <- paste(head(of, 1))
data <- read.table(file=i, header=T)
y<-data$PHI
num<-length(y)
index1<-c(1:num_sample)[y==1]
index2<-c(1:num_sample)[y==0]
gen<-as.matrix(data[,-c(1:2)])
source("pcc.R") # a function for my use
out<- fpc_func(gen,num,index1,index2)
out1<-as.data.frame(out)
id1<-data[,2]
id<- as.data.frame(iid1)
out2<-cbind(iid1,out1)
colnames(out2)[2] <- of
}
write.table(out2, file="ALL.txt", append=T, col.names=T, row.names=F,
quote=F, sep = "\t")
#######################
I can do it for each file separately but can't figure out how to store the
results in one file.
If I do it for each file, the output I get
For ABCD.raw
id? ABCD
1???? 0.11
2??? -0.24
3???? 2.18
........
......
For EFGH.raw
id? ABCD
1???? -0.18
2???? -1.33
3????? 0.88
........
......
?etc.
I would like to get the results in one file
id? ABCD?? EFGH?? IJKL .........
1???? 0.11???? -0.18??? ...........
2??? -0.24??? -1.33??? ...........
3???? 2.18???? 0.88?? .......
........
......
Thanks,
Debs
Jean V Adams
2012-Jun-28 19:05 UTC
[R] Storing results in a single file after looping over all files
Without providing data and code that I can actually run and test, I can only take a guess at how to solve this. See my edits and suggestions in the code, below. Hope it helps. Jean Debs Majumdar <debs_stata@yahoo.com> wrote on 06/28/2012 12:49:40 AM:> Hi All, > > I have a whole lot of *.raw files in my working folder and I am > doing the same analysis on each of those and want to save all the > results in a single file. I am making some mistake here and can't > figure out how to solve it. > > > Say, the *.raw files are ABCD.raw, EFGH.raw, IJKL.raw ... > > > The files are of this format > > ID PHI aa1 aa2 aa3 .... > > 1 1 1.3 2.0 1.0 > > 2 0 1.5 NA 0.9 > > 3 1 0.1 0.2 1.5 > > ...... > .. > > > My code is as follows: > > files <- list.files(pattern="*.raw")# before your for() loop, create an empty list to put your results in outs <- vector("list", length(files))> for(i in files){ > of <- strsplit(i, "\\.")[[1]] > of <- paste(head(of, 1)) > > data <- read.table(file=i, header=T) > y<-data$PHI > > num<-length(y) > index1<-c(1:num_sample)[y==1] > index2<-c(1:num_sample)[y==0] > gen<-as.matrix(data[,-c(1:2)]) > source("pcc.R") # a function for my use > out<- fpc_func(gen,num,index1,index2) > > > out1<-as.data.frame(out) > id1<-data[,2] > id<- as.data.frame(iid1)# then in the next two lines, change out2 to outs[[i]] outs[[i]]<-cbind(iid1,out1) colnames(outs[[i]])[2] <- of> out2<-cbind(iid1,out1) > colnames(out2)[2] <- of > }# after the for() loop, combine all data.frames by column out2 <- do.call(cbind, outs)> write.table(out2, file="ALL.txt", append=T, col.names=T, > row.names=F, quote=F, sep = "\t") > > ####################### > > I can do it for each file separately but can't figure out how to > store the results in one file. > > If I do it for each file, the output I get > > For ABCD.raw > > id ABCD > > 1 0.11 > 2 -0.24 > > 3 2.18 > > ........ > ...... > > > > For EFGH.raw > > id ABCD > > 1 -0.18 > 2 -1.33 > > 3 0.88 > > ........ > ...... > > etc. > > I would like to get the results in one file > > id ABCD EFGH IJKL ......... > > 1 0.11 -0.18 ........... > 2 -0.24 -1.33 ........... > > 3 2.18 0.88 ....... > > ........ > ...... > > Thanks, > > Debs[[alternative HTML version deleted]]
Rui Barradas
2012-Jun-28 20:26 UTC
[R] Storing results in a single file after looping over all files
Hello,
You should post your data in a readable format.
Use ?dput.
Now, for your question, start by putting your results in a list.
abcd <- read.table(text="
id ABCD
1 0.11
2 -0.24
3 2.18
", header=TRUE)
efgh <- read.table(text="
id EFGH
1 -0.18
2 -1.33
3 0.88
", header=TRUE)
ijkl <- read.table(text="
id IJKL
1 0.81
2 -1.03
3 1.38
", header=TRUE)
(df.list <- list(abcd, efgh, ijkl))
You now have two ways.
1. Assuming that all data.frames have the same 'id' values:
result <-do.call(cbind, list(df.list[[1]], lapply(df.list[-1], `[`, 2)))
2. If they do not, or if it's not certain:
result <- df.list[[1]]
lapply(df.list[-1], function(x){ result <<- merge(result, x); NULL })
And all you have to do is write.table().
Hope this helps,
Rui Barradas
Em 28-06-2012 06:49, Debs Majumdar escreveu:> Hi All,
>
> I have a whole lot of *.raw files in my working folder and I am doing the
same analysis on each of those and want to save all the results in a single
file. I am making some mistake here and can't figure out how to solve it.
>
>
> Say, the *.raw files are ABCD.raw, EFGH.raw, IJKL.raw ...
>
>
> The files are of this format
>
> ID PHI aa1 aa2 aa3 ....
>
> 1 1 1.3 2.0 1.0
>
> 2 0 1.5 NA 0.9
>
> 3 1 0.1 0.2 1.5
>
> ......
> ..
>
>
> My code is as follows:
>
> files <- list.files(pattern="*.raw")
> for(i in files){
> of <- strsplit(i, "\\.")[[1]]
> of <- paste(head(of, 1))
>
> data <- read.table(file=i, header=T)
> y<-data$PHI
>
> num<-length(y)
> index1<-c(1:num_sample)[y==1]
> index2<-c(1:num_sample)[y==0]
> gen<-as.matrix(data[,-c(1:2)])
> source("pcc.R") # a function for my use
> out<- fpc_func(gen,num,index1,index2)
>
>
> out1<-as.data.frame(out)
> id1<-data[,2]
> id<- as.data.frame(iid1)
> out2<-cbind(iid1,out1)
> colnames(out2)[2] <- of
> }
> write.table(out2, file="ALL.txt", append=T, col.names=T,
row.names=F, quote=F, sep = "\t")
>
> #######################
>
> I can do it for each file separately but can't figure out how to store
the results in one file.
>
> If I do it for each file, the output I get
>
> For ABCD.raw
>
> id ABCD
>
> 1 0.11
> 2 -0.24
>
> 3 2.18
>
> ........
> ......
>
>
>
> For EFGH.raw
>
> id ABCD
>
> 1 -0.18
> 2 -1.33
>
> 3 0.88
>
> ........
> ......
>
> etc.
>
> I would like to get the results in one file
>
> id ABCD EFGH IJKL .........
>
> 1 0.11 -0.18 ...........
> 2 -0.24 -1.33 ...........
>
> 3 2.18 0.88 .......
>
> ........
> ......
>
> Thanks,
>
> Debs
>
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
Debs Majumdar
2012-Jun-28 22:35 UTC
[R] Storing results in a single file after looping over all files
Hi, ? I can upload the toy datasets and code but am unsure about a good hosting site. Suggestions? I got the following error after the following command: out2 <- do.call(cbind, outs) Error in data.frame(..., check.names = FALSE) : ? arguments imply differing number of rows: 0, 750 ### I have 750 unique IDs in each of the raw datasets. Thanks, Debs ________________________________ From: Jean V Adams <jvadams at usgs.gov> To: Debs Majumdar <debs_stata at yahoo.com> Cc: "r-help at r-project.org" <r-help at r-project.org> Sent: Thursday, June 28, 2012 12:05 PM Subject: Re: [R] Storing results in a single file after looping over all files Without providing data and code that I can actually run and test,?? I can only take a guess at how to solve this. See my edits and suggestions in the code, below. Hope it helps. Jean Debs Majumdar <debs_stata at yahoo.com> wrote on 06/28/2012 12:49:40 AM:> Hi All, > > I have a whole lot of *.raw files in my working folder and I am > doing the same analysis on each of those and want to save all the > results in a single file. I am making some mistake here and can't > figure out how to solve it. > > > Say, the *.raw files are ABCD.raw, EFGH.raw, IJKL.raw ... > > > The files are of this format > > ID PHI?? aa1? aa2? aa3 .... > > 1??? 1???? 1.3?? 2.0??1.0> > 2??? 0???? 1.5?? NA??0.9> > 3??? 1???? 0.1?? 0.2??1.5> > ...... > .. > > > My code is as follows: > > files <- list.files(pattern="*.raw")# before your for() loop, create an empty list to put your results in outs <- vector("list", length(files))> for(i in files){ > of <- strsplit(i, "\\.")[[1]] > of <- paste(head(of, 1)) > > data <- read.table(file=i, header=T) > y<-data$PHI > > num<-length(y) > index1<-c(1:num_sample)[y==1] > index2<-c(1:num_sample)[y==0] > gen<-as.matrix(data[,-c(1:2)]) > source("pcc.R") # a function for my use > out<- fpc_func(gen,num,index1,index2) > > > out1<-as.data.frame(out) > id1<-data[,2] > id<- as.data.frame(iid1)# then in the next two lines, change out2 to outs[[i]] outs[[i]]<-cbind(iid1,out1)?? colnames(outs[[i]])[2] <- of??> out2<-cbind(iid1,out1) > colnames(out2)[2] <- of > }# after the for() loop, combine all data.frames by column out2 <- do.call(cbind, outs)> write.table(out2, file="ALL.txt", append=T, col.names=T, > row.names=F, quote=F, sep = "\t") > > ####################### > > I can do it for each file separately but can't figure out how to > store the results in one file. > > If I do it for each file, the output I get > > For ABCD.raw > > id? ABCD > > 1???? 0.11 > 2??? -0.24 > > 3???? 2.18 > > ........ > ...... > > > > For EFGH.raw > > id? ABCD > > 1???? -0.18 > 2???? -1.33 > > 3????? 0.88 > > ........ > ...... > > ?etc. > > I would like to get the results in one file > > id? ABCD?? EFGH?? IJKL ......... > > 1???? 0.11???? -0.18???...........> 2??? -0.24??? -1.33???...........> > 3???? 2.18???? 0.88??.......> > ........ > ...... > > Thanks, > > Debs