R help - Sep 2009 - Average over data sets

Hello,

I have a number of files output1.dat, output2.dat, ... , output20.dat,  
each of which monitors several variables over a fixed number of  
timepoints. From this I want to create a data frame which contains the  
mean value between all files, for each timepoint and each variable.

The code below works, but it seems like I should be able to do the  
second part without a for loop. I played with sapply(myList, mean),  
but that seems to take the mean between time points and files, rather  
than just between files.

#Number of files to calculate mean value between
numberOfRuns = 20;
myList = list();
for (i in 1:numberOfRuns) {
	#Read in file
	fileName = paste("output", i, ".dat", sep="");
	myData = read.table(fileName, header=TRUE);
	#Append data frame to list
	myList[[i]] = myData;
}

#Create variable to store data means
myAverage = myList[[1]]/numberOfRuns;

for (i in 2:numberOfRuns) {
	myAverage = myAverage + myList[[i]]/numberOfRuns;
}

Is a list of data frames a sensible structure to store this or should  
I use an array?

Any pointers gratefully received.

Ed Long

Why not create a single data frame with all the results and then you
can easily use tapply/sapply/aggregate/reshape package

# combine into a single DF
combinedDF <- do.call(rbind, MyList)


On Wed, Sep 2, 2009 at 8:55 AM, Ed Long<e.long at ucl.ac.uk>
wrote:
> Hello,
>
> I have a number of files output1.dat, output2.dat, ... , output20.dat, each
> of which monitors several variables over a fixed number of timepoints. From
> this I want to create a data frame which contains the mean value between
all
> files, for each timepoint and each variable.
>
> The code below works, but it seems like I should be able to do the second
> part without a for loop. I played with sapply(myList, mean), but that seems
> to take the mean between time points and files, rather than just between
> files.
>
> #Number of files to calculate mean value between
> numberOfRuns = 20;
> myList = list();
> for (i in 1:numberOfRuns) {
> ? ? ? ?#Read in file
> ? ? ? ?fileName = paste("output", i, ".dat",
sep="");
> ? ? ? ?myData = read.table(fileName, header=TRUE);
> ? ? ? ?#Append data frame to list
> ? ? ? ?myList[[i]] = myData;
> }
>
> #Create variable to store data means
> myAverage = myList[[1]]/numberOfRuns;
>
> for (i in 2:numberOfRuns) {
> ? ? ? ?myAverage = myAverage + myList[[i]]/numberOfRuns;
> }
>
> Is a list of data frames a sensible structure to store this or should I use
> an array?
>
> Any pointers gratefully received.
>
> Ed Long
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>


-- 
Jim Holtman
Cincinnati, OH
+1 513 646 9390

What is the problem that you are trying to solve?

Here is a solution assuming that all files have the same structure and a
variable TimePoint which contains the time info.

CombinedData <- do.call(rbind, lapply(seq_len(20), function(i){
	fileName <- paste("output", i, ".dat", sep="")
	read.table(fileName, header=TRUE)
}))
library(plyr)
ddply(CombinedData, "TimePoint", colMeans) 


#another option
library(reshape)
recast(CombinedData, TimePoint + variable ~ ., id.var = TimePoint, fun mean)

HTH,

Thierry

------------------------------------------------------------------------
----
ir. Thierry Onkelinx
Instituut voor natuur- en bosonderzoek / Research Institute for Nature
and Forest
Cel biometrie, methodologie en kwaliteitszorg / Section biometrics,
methodology and quality assurance
Gaverstraat 4
9500 Geraardsbergen
Belgium
tel. + 32 54/436 185
Thierry.Onkelinx at inbo.be
www.inbo.be

To call in the statistician after the experiment is done may be no more
than asking him to perform a post-mortem examination: he may be able to
say what the experiment died of.
~ Sir Ronald Aylmer Fisher

The plural of anecdote is not data.
~ Roger Brinner

The combination of some data and an aching desire for an answer does not
ensure that a reasonable answer can be extracted from a given body of
data.
~ John Tukey

-----Oorspronkelijk bericht-----
Van: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org]
Namens Ed Long
Verzonden: woensdag 2 september 2009 14:55
Aan: r-help at r-project.org
Onderwerp: [R] Average over data sets

Hello,

I have a number of files output1.dat, output2.dat, ... , output20.dat,
each of which monitors several variables over a fixed number of
timepoints. From this I want to create a data frame which contains the
mean value between all files, for each timepoint and each variable.

The code below works, but it seems like I should be able to do the
second part without a for loop. I played with sapply(myList, mean), but
that seems to take the mean between time points and files, rather than
just between files.

#Number of files to calculate mean value between numberOfRuns = 20;
myList = list(); for (i in 1:numberOfRuns) {
	#Read in file
	fileName = paste("output", i, ".dat", sep="");
	myData = read.table(fileName, header=TRUE);
	#Append data frame to list
	myList[[i]] = myData;
}

#Create variable to store data means
myAverage = myList[[1]]/numberOfRuns;

for (i in 2:numberOfRuns) {
	myAverage = myAverage + myList[[i]]/numberOfRuns; }

Is a list of data frames a sensible structure to store this or should I
use an array?

Any pointers gratefully received.

Ed Long

______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Druk dit bericht a.u.b. niet onnodig af.
Please do not print this message unnecessarily.

Dit bericht en eventuele bijlagen geven enkel de visie van de schrijver weer 
en binden het INBO onder geen enkel beding, zolang dit bericht niet bevestigd is
door een geldig ondertekend document. The views expressed in  this message 
and any annex are purely those of the writer and may not be regarded as stating 
an official position of INBO, as long as the message is not confirmed by a duly 
signed document.