Hi all,
I would like to set up an iterative read & write sequence to avoid
reading and writing each file one at a time.
Hundreds of data sets to re-calculate.? The code I have works well
individually, but would like to set up an iterative read, calculate and
write changing the input and output file names each iteration.
I? think I have read that there is an R? feature using "Pipes" and %
that may help but have not figured out how to do this.
What I am seeking is a way to read a list of TXT files, calculate the
values and write the results in an iterative manner to avoid manually
reading each file individually
Suggestions?
Tnx,
Bruce*
*
*# This works*
*#read in a data file
*
*>Corbre_2 <- read.table("E:/Acoustic Parameters/~Emballonuridae/Data
by
genus/~Corbre/Corbre_2.txt",header=T,sep="\t",quote="")*
*# create and calculate a new variable for time between call pusles
*
*Corbre_2 [c("TBC")] <-
recode.variables(Corbre_2[c("TBC")] , "0 -> NA;")*
*# Create new BW variable for band width then change data file name*
*BW<-within(Corbre_2,BW<-Fmax-Fmin)*
*# Create 2 new variables for fundamental H1 and third harmonic H3
frequencies
*
*#based on Fc that is generally loudest at the 2^nd harmonic.*
*BW<-within(BW,FcH1<-Fc*.5)*
*BW<-within(BW,FcH3<-FcH1*3)*
*#rename to a DF to calculate the stats
*
*BatStats<-BW*
*BatStats<-Deducer::descriptive.table (vars = d
(Dur,TBC,Fmin,Fmax,BW,Fmean,Fk,FcH1,Fc,FcH3,Sc,Pmc),data=
BatStats,func.names =c("Valid
N","Minimum","Maximum","Mean","St.
Deviation"),func.additional= list(p10=function(x) quantile(x, c(0.10),
na.rm=TRUE),p25=function(x) quantile(x, c(0.25),
na.rm=TRUE),p75=function(x) quantile(x,
c(0.75),na.rm=TRUE),p90=function(x) quantile(x, c(0.90),na.rm=TRUE)))*
*## Write the results and change output name and directory for the
calculated stats.*
*write.csv(BatStats,file="C:\\=Bat data working\\Acoustic
Parameters\\Corbre_Stats.csv")*
--
Bruce W. Miller, PhD.
Neotropical bat risk assessments
Conservation Fellow - Wildlife Conservation Society
If we lose the bats, we may lose much of the tropical vegetation and the lungs
of the planet
Using acoustic sampling to identify and map species distributions for >25
years.
Providing free Interactive identification keys and call fact sheets for the
vocal signatures of New World Bats
[[alternative HTML version deleted]]
Suggestions? Yes. Don't do this. Instead, wrap your code in a function, use lapply() to apply it over your list of files, returning all results in one list, and then save the list in an .rdata file. Also, in future, please post in plain text, as html tends to get mangled on this plain text list . Bert On Tue, Apr 17, 2018, 4:20 AM Neotropical bat risk assessments < neotropical.bats at gmail.com> wrote:> Hi all, > > I would like to set up an iterative read & write sequence to avoid > reading and writing each file one at a time. > Hundreds of data sets to re-calculate. The code I have works well > individually, but would like to set up an iterative read, calculate and > write changing the input and output file names each iteration. > > I think I have read that there is an R feature using "Pipes" and % > that may help but have not figured out how to do this. > What I am seeking is a way to read a list of TXT files, calculate the > values and write the results in an iterative manner to avoid manually > reading each file individually > > Suggestions? > > Tnx, > Bruce* > * > > *# This works* > > *#read in a data file > * > > *>Corbre_2 <- read.table("E:/Acoustic Parameters/~Emballonuridae/Data by > genus/~Corbre/Corbre_2.txt",header=T,sep="\t",quote="")* > > *# create and calculate a new variable for time between call pusles > * > > *Corbre_2 [c("TBC")] <- recode.variables(Corbre_2[c("TBC")] , "0 -> NA;")* > > *# Create new BW variable for band width then change data file name* > > *BW<-within(Corbre_2,BW<-Fmax-Fmin)* > > *# Create 2 new variables for fundamental H1 and third harmonic H3 > frequencies > * > > *#based on Fc that is generally loudest at the 2^nd harmonic.* > > *BW<-within(BW,FcH1<-Fc*.5)* > > *BW<-within(BW,FcH3<-FcH1*3)* > > *#rename to a DF to calculate the stats > * > > *BatStats<-BW* > > *BatStats<-Deducer::descriptive.table (vars = d > (Dur,TBC,Fmin,Fmax,BW,Fmean,Fk,FcH1,Fc,FcH3,Sc,Pmc),data> BatStats,func.names =c("Valid N","Minimum","Maximum","Mean","St. > Deviation"),func.additional= list(p10=function(x) quantile(x, c(0.10), > na.rm=TRUE),p25=function(x) quantile(x, c(0.25), > na.rm=TRUE),p75=function(x) quantile(x, > c(0.75),na.rm=TRUE),p90=function(x) quantile(x, c(0.90),na.rm=TRUE)))* > > *## Write the results and change output name and directory for the > calculated stats.* > > *write.csv(BatStats,file="C:\\=Bat data working\\Acoustic > Parameters\\Corbre_Stats.csv")* > > > > -- > Bruce W. Miller, PhD. > Neotropical bat risk assessments > Conservation Fellow - Wildlife Conservation Society > > If we lose the bats, we may lose much of the tropical vegetation and the > lungs of the planet > > Using acoustic sampling to identify and map species distributions for >25 > years. > > Providing free Interactive identification keys and call fact sheets for > the vocal signatures of New World Bats > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >On Apr 17, 2018 4:20 AM, "Neotropical bat risk assessments" < neotropical.bats at gmail.com> wrote: Hi all, I would like to set up an iterative read & write sequence to avoid reading and writing each file one at a time. Hundreds of data sets to re-calculate. The code I have works well individually, but would like to set up an iterative read, calculate and write changing the input and output file names each iteration. I think I have read that there is an R feature using "Pipes" and % that may help but have not figured out how to do this. What I am seeking is a way to read a list of TXT files, calculate the values and write the results in an iterative manner to avoid manually reading each file individually Suggestions? Tnx, Bruce* * *# This works* *#read in a data file * *>Corbre_2 <- read.table("E:/Acoustic Parameters/~Emballonuridae/Data by genus/~Corbre/Corbre_2.txt",header=T,sep="\t",quote="")* *# create and calculate a new variable for time between call pusles * *Corbre_2 [c("TBC")] <- recode.variables(Corbre_2[c("TBC")] , "0 -> NA;")* *# Create new BW variable for band width then change data file name* *BW<-within(Corbre_2,BW<-Fmax-Fmin)* *# Create 2 new variables for fundamental H1 and third harmonic H3 frequencies * *#based on Fc that is generally loudest at the 2^nd harmonic.* *BW<-within(BW,FcH1<-Fc*.5)* *BW<-within(BW,FcH3<-FcH1*3)* *#rename to a DF to calculate the stats * *BatStats<-BW* *BatStats<-Deducer::descriptive.table (vars = d (Dur,TBC,Fmin,Fmax,BW,Fmean,Fk,FcH1,Fc,FcH3,Sc,Pmc),dataBatStats,func.names =c("Valid N","Minimum","Maximum","Mean","St. Deviation"),func.additional= list(p10=function(x) quantile(x, c(0.10), na.rm=TRUE),p25=function(x) quantile(x, c(0.25), na.rm=TRUE),p75=function(x) quantile(x, c(0.75),na.rm=TRUE),p90=function(x) quantile(x, c(0.90),na.rm=TRUE)))* *## Write the results and change output name and directory for the calculated stats.* *write.csv(BatStats,file="C:\\=Bat data working\\Acoustic Parameters\\Corbre_Stats.csv")* -- Bruce W. Miller, PhD. Neotropical bat risk assessments Conservation Fellow - Wildlife Conservation Society If we lose the bats, we may lose much of the tropical vegetation and the lungs of the planet Using acoustic sampling to identify and map species distributions for >25 years. Providing free Interactive identification keys and call fact sheets for the vocal signatures of New World Bats [[alternative HTML version deleted]] ______________________________________________ R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]]
Dear Bruce,
I'd write a function that take filenames as arguments. It should read
the file, do the calculations and write that output to another file
which can be calculated from the input. You can get all input files in
a directory with e.g. list.files(). Then you can simply loop over the
inputs. Here is some pseudo code.
my_summary <- function(input) {
raw <- read.table(input)
summarised <- summary(raw)
output <- calculate_output(input)
write.csv(summarised, output)
}
inputs <- list.files("raw/data/path")
sapply(inputs, my_summary)
Best regards,
ir. Thierry Onkelinx
Statisticus / Statistician
Vlaamse Overheid / Government of Flanders
INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE
AND FOREST
Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance
thierry.onkelinx at inbo.be
Havenlaan 88 bus 73, 1000 Brussel
www.inbo.be
///////////////////////////////////////////////////////////////////////////////////////////
To call in the statistician after the experiment is done may be no
more than asking him to perform a post-mortem examination: he may be
able to say what the experiment died of. ~ Sir Ronald Aylmer Fisher
The plural of anecdote is not data. ~ Roger Brinner
The combination of some data and an aching desire for an answer does
not ensure that a reasonable answer can be extracted from a given body
of data. ~ John Tukey
///////////////////////////////////////////////////////////////////////////////////////////
2018-04-17 13:20 GMT+02:00 Neotropical bat risk assessments
<neotropical.bats at gmail.com>:> Hi all,
>
> I would like to set up an iterative read & write sequence to avoid
> reading and writing each file one at a time.
> Hundreds of data sets to re-calculate. The code I have works well
> individually, but would like to set up an iterative read, calculate and
> write changing the input and output file names each iteration.
>
> I think I have read that there is an R feature using "Pipes"
and %
> that may help but have not figured out how to do this.
> What I am seeking is a way to read a list of TXT files, calculate the
> values and write the results in an iterative manner to avoid manually
> reading each file individually
>
> Suggestions?
>
> Tnx,
> Bruce*
> *
>
> *# This works*
>
> *#read in a data file
> *
>
> *>Corbre_2 <- read.table("E:/Acoustic
Parameters/~Emballonuridae/Data by
>
genus/~Corbre/Corbre_2.txt",header=T,sep="\t",quote="")*
>
> *# create and calculate a new variable for time between call pusles
> *
>
> *Corbre_2 [c("TBC")] <-
recode.variables(Corbre_2[c("TBC")] , "0 -> NA;")*
>
> *# Create new BW variable for band width then change data file name*
>
> *BW<-within(Corbre_2,BW<-Fmax-Fmin)*
>
> *# Create 2 new variables for fundamental H1 and third harmonic H3
> frequencies
> *
>
> *#based on Fc that is generally loudest at the 2^nd harmonic.*
>
> *BW<-within(BW,FcH1<-Fc*.5)*
>
> *BW<-within(BW,FcH3<-FcH1*3)*
>
> *#rename to a DF to calculate the stats
> *
>
> *BatStats<-BW*
>
> *BatStats<-Deducer::descriptive.table (vars = d
> (Dur,TBC,Fmin,Fmax,BW,Fmean,Fk,FcH1,Fc,FcH3,Sc,Pmc),data>
BatStats,func.names =c("Valid
N","Minimum","Maximum","Mean","St.
> Deviation"),func.additional= list(p10=function(x) quantile(x, c(0.10),
> na.rm=TRUE),p25=function(x) quantile(x, c(0.25),
> na.rm=TRUE),p75=function(x) quantile(x,
> c(0.75),na.rm=TRUE),p90=function(x) quantile(x, c(0.90),na.rm=TRUE)))*
>
> *## Write the results and change output name and directory for the
> calculated stats.*
>
> *write.csv(BatStats,file="C:\\=Bat data working\\Acoustic
> Parameters\\Corbre_Stats.csv")*
>
>
>
> --
> Bruce W. Miller, PhD.
> Neotropical bat risk assessments
> Conservation Fellow - Wildlife Conservation Society
>
> If we lose the bats, we may lose much of the tropical vegetation and the
lungs of the planet
>
> Using acoustic sampling to identify and map species distributions for
>25 years.
>
> Providing free Interactive identification keys and call fact sheets for the
vocal signatures of New World Bats
>
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.