Hi I'm trying to speed my loop up. Any Suggestions?? At the moment it takes
a
few days to run.
THE CODE
---------------------------------------------------------------------------------------------------
for(i in 1:11) {
for (j in 3:12) {
for (k in 1:273107) {
y[k,1] <- x[i,j,k]
print(y)
Rainfall_dataset <- read.table("1km_grid_nzmg.csv",
header=TRUE, sep=",", na.strings="NA",
dec=".", strip.white=TRUE)
t <- Rainfall_dataset
print(t)
p<-cbind(t,y)
#y<-cbind(t,y)
#print(y)
}
# This where you write out array y to a csv file
# Save output as a csv file
site <- paste("site",i,"-",j)
csvfile <- paste(site,sep=".","csv");
print(csvfile)
path<- paste("c:/Data/",csvfile)
print(path)
write.csv(p,file=path, row.names = TRUE)
}
}
--
View this message in context:
http://r.789695.n4.nabble.com/Speeding-nested-loops-up-tp3751485p3751485.html
Sent from the R help mailing list archive at Nabble.com.
General suggestions: avoid cbind() and avoid accessing data frames. Convert
data frames to matrices before accessing them. Also, why do you print? You
don't really want to print(t) for every iteration of the loop, do you? Also
avoid defining elements within the loop that need to be defined only once (
such as path<- paste("c:/Data/",csvfile) ).
For more specific help it would help if you provided a minimally
self-contained example (one that we can copy-paste in the R prompt) as
requested in the posting guide.
HTH,
Daniel
mark_horo wrote:>
> Hi I'm trying to speed my loop up. Any Suggestions?? At the moment it
> takes a few days to run.
>
> THE CODE
>
---------------------------------------------------------------------------------------------------
> for(i in 1:11) {
> for (j in 3:12) {
> for (k in 1:273107) {
> y[k,1] <- x[i,j,k]
> print(y)
> Rainfall_dataset <- read.table("1km_grid_nzmg.csv",
> header=TRUE, sep=",", na.strings="NA",
dec=".", strip.white=TRUE)
>
> t <- Rainfall_dataset
> print(t)
>
> p<-cbind(t,y)
> #y<-cbind(t,y)
> #print(y)
> }
> # This where you write out array y to a csv file
> # Save output as a csv file
> site <- paste("site",i,"-",j)
>
> csvfile <- paste(site,sep=".","csv");
print(csvfile)
> path<- paste("c:/Data/",csvfile)
> print(path)
> write.csv(p,file=path, row.names = TRUE)
> }
> }
>
--
View this message in context:
http://r.789695.n4.nabble.com/Speeding-nested-loops-up-tp3751485p3751542.html
Sent from the R help mailing list archive at Nabble.com.
On Aug 17, 2011, at 8:14 PM, mark_horo wrote:> Hi I'm trying to speed my loop up. Any Suggestions?? At the moment > it takes a > few days to run. > > THE CODEWe are not surprised. Rainfall_dataset <- read.table("1km_grid_nzmg.csv", header=TRUE, sep=",", na.strings="NA", dec=".", strip.white=TRUE)> --------------------------------------------------------------------------------------------------- > for(i in 1:11) { > for (j in 3:12) {# What is x ??? Where is y dimensioned??? # whatever the answers, this would be faster ---> y[ ,1] <- x[i,j, ]> > for (k in 1:273107) { > # and this is no longer needed in the inner loop y[k,1] <- x[i,j,k]> print(y) > > # t <- Rainfall_dataset > # print(t) > p<-cbind(t,y)???? You are just writing out a set of vectors as the first column to identical copies of the entire RainfalL_dataset?> #y<-cbind(t,y) Huh??? > #print(y) > } > # This where you write out array y to a csv file > # Save output as a csv file > site <- paste("site",i,"-",j) > > csvfile <- paste(site,sep=".","csv"); print(csvfile) > path<- paste("c:/Data/",csvfile) > print(path) > write.csv(p,file=path, row.names = TRUE) > } > } >To me this looks like a 11*10* 273107 == 30MB set of files with no real added information. Surely a database solution must be better. -- David Winsemius, MD West Hartford, CT
anytime my code starts to take more than a couple of minutes to run what might look like a simple set of commands, I use Rprof to see where the time goes. Run this on your script and you might get some insight into the problem area. If it ran for an hour, I would defintely take a deep dive into it. Sent from my iPad On Aug 17, 2011, at 20:14, mark_horo <markknightnz at yahoo.co.nz> wrote:> Hi I'm trying to speed my loop up. Any Suggestions?? At the moment it takes a > few days to run. > > THE CODE > --------------------------------------------------------------------------------------------------- > for(i in 1:11) { > for (j in 3:12) { > for (k in 1:273107) { > y[k,1] <- x[i,j,k] > print(y) > Rainfall_dataset <- read.table("1km_grid_nzmg.csv", > header=TRUE, sep=",", na.strings="NA", dec=".", strip.white=TRUE) > > t <- Rainfall_dataset > print(t) > > p<-cbind(t,y) > #y<-cbind(t,y) > #print(y) > } > # This where you write out array y to a csv file > # Save output as a csv file > site <- paste("site",i,"-",j) > > csvfile <- paste(site,sep=".","csv"); print(csvfile) > path<- paste("c:/Data/",csvfile) > print(path) > write.csv(p,file=path, row.names = TRUE) > } > } > > -- > View this message in context: http://r.789695.n4.nabble.com/Speeding-nested-loops-up-tp3751485p3751485.html > Sent from the R help mailing list archive at Nabble.com. > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.