I have over 8000 time series that I need to analyze and forecast. Running 1500 takes over 2 hours using just ETS, let alone Holt-Winters and ARIMA. So I am looking at ways in shrinking the time to generate a 2 year forecast. The code I am using successfully to run through the time series sequentially is below. The essence of the code being reading data from multiple CSV files, 1 per data set, that contain up to 5 years of historical sales by item. I parse each file out by item, generate a time-series for each item, fit the ETS model by item, generate a 24 months forecast by item, add the item number to the forecast, and write the forecast to an Excel file. I'm looking for guidance in two areas: * Reading the raw data in from Excel which is in the form: d1 d2 d3 d4 ... series 1 v11 v12 v13 v14 series 2 v21 v22 v23 v24 . . * Using parallel processing to analyze the data more quickly using several cores. I have tried to use doParallel at the item level, but without success. I have annotated the code to show where I tried to insert the %dopar% aspects. # store the current directory initial.dir<-getwd() # change to the new directory setwd("~/R") # load the necessary libraries require(TTR) require(forecast) require(xlsx) #require(doParallel) #cl <- makeCluster(3) #registerDoSNOW(cl) #chunks <- getDoParWorkers() # output plots to a file pdf("R Plots.pdf") # set the output file sink(file = "R Output.out", type = c("output")) # load the dataset files <- c("3MH", "6MH", "12MH") for (j in 1:3) { title <- paste("\n\n\n Evaluation of", files[j], " - Started at", date(), "\n\n\n") cat(title) History <- read.csv(paste(files[j],"csv", sep=".")) # output forecast to XLSX outwb <- createWorkbook() sheet <- createSheet(outwb, sheetName = paste(files[j], " - ETS")) Item <- unique(unlist(History$Item)) for (i in 1:length(Item)) # I tried using r <- foreach(i=1:length(Item) , .combine='rbind') %dopar% at this level { title <- paste("Evaluation of item ", Item[i], "-", i, "of", length(Item),"\n") cat(title) data <- subset(History, Item == Item[i]) dates <- unique(unlist(data$Date)) d <- as.Date(dates, format("%d/%m/%Y")) data.ts <- ts(data$Volume, frequency=12, start=c(as.numeric(format(d[1],"%Y")), as.numeric(format(d[1],"%m")))) #try(plot(decompose(data.ts))) #acf(data.ts) try(data.ets <- ets(data.ts)) try(forecast.ets <- forecast.ets(data.ets, h=24)) IL <- c(Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i]) ets.df <- data.frame(forecast.ets) ets.df$Item <- IL r <- 24*(i-1)+2 addDataFrame(ets.df, sheet, col.names=FALSE, startRow=r) } title <- paste("\n\n\n Evaluation of", files[j], " - Completed at", date(), "\n\n\n") cat(title) saveWorkbook(outwb, paste(files[j],"xlsx",sep='.')) } # close the output file sink() dev.off() #stopCluster(cl) # change back to the original directory setwd(initial.dir) Trevor Miles Vice President, Thought Leadership [http://www.kinaxis.com/email-signature/images/logo-kinaxis.png]<http://www.kinaxis.com> O: +1.613.907.7611 | M: +1.647.248.6269 | T: @MilesAhead<https://twitter.com/milesahead> | L: ca.linkedin.com/in/trevormiles<http://ca.linkedin.com/in/trevormiles> [Kinexions '14]<http://kinexions.kinaxis.com> [http://www2.kinaxis.com/email-signature/images/social-icon-twitter.png]<http://twitter.com/kinaxis> [http://www2.kinaxis.com/email-signature/images/social-icon-facebook.png] <http://www.facebook.com/Kinaxis> [http://www2.kinaxis.com/email-signature/images/social-icon-linkedin.png] <http://www.linkedin.com/company/kinaxis> [http://www2.kinaxis.com/email-signature/images/social-icon-community.png] <https://community.kinaxis.com> Confidential. This email and any attachments hereto may contain private, confidential, and privileged material for the sole use of the addressee. Any review, copying, or distribution of this email (or any attachments thereto) by others is strictly prohibited. If you are not the intended recipient, please return this email to the sender immediately and permanently delete the original and any copies of this email and any of its attachments. Thank you. [[alternative HTML version deleted]]