R help - Aug 2014 - Time series analysis for a large number of series

I have over 8000 time series that I need to analyze and forecast. Running 1500
takes over 2 hours using just ETS, let alone Holt-Winters and ARIMA. So I am
looking at ways in shrinking the time to generate a 2 year forecast.

The code I am using successfully to run through the time series sequentially is
below. The essence of the code being reading data from multiple CSV files, 1 per
data set, that contain up to 5 years of historical sales by item. I parse each
file out by item, generate a time-series for each item, fit the ETS model by
item, generate a 24 months forecast by item, add the item number to the
forecast, and write the forecast to an Excel file.

I'm looking for guidance in two areas:

*         Reading the raw data in from Excel which is in the form:
             d1    d2    d3   d4    ...
series 1  v11   v12  v13  v14
series 2  v21   v22  v23  v24
.
.

*         Using parallel processing to analyze the data more quickly using
several cores.

I have tried to use doParallel at the item level, but without success. I have
annotated the code to show where I tried to insert the %dopar% aspects.

# store the current directory
initial.dir<-getwd()
# change to the new directory
setwd("~/R")
# load the necessary libraries
require(TTR)
require(forecast)
require(xlsx)

#require(doParallel)
#cl <- makeCluster(3)
#registerDoSNOW(cl)
#chunks <- getDoParWorkers()

# output plots to a file
pdf("R Plots.pdf")
# set the output file
sink(file = "R Output.out", type = c("output"))

# load the dataset
files <- c("3MH", "6MH", "12MH")
for (j in 1:3)
{
  title <- paste("\n\n\n Evaluation of", files[j], " - Started
at", date(), "\n\n\n")
  cat(title)

  History <- read.csv(paste(files[j],"csv", sep="."))

  # output forecast to XLSX
  outwb <- createWorkbook()
  sheet <- createSheet(outwb, sheetName = paste(files[j], " -
ETS"))
  Item <- unique(unlist(History$Item))

  for (i in 1:length(Item))  # I tried using r <- foreach(i=1:length(Item) ,
.combine='rbind') %dopar% at this level
  {
    title <- paste("Evaluation of item ", Item[i], "-",
i, "of", length(Item),"\n")
    cat(title)
    data <- subset(History, Item == Item[i])
    dates <- unique(unlist(data$Date))
    d <- as.Date(dates, format("%d/%m/%Y"))
    data.ts <- ts(data$Volume, frequency=12,
start=c(as.numeric(format(d[1],"%Y")),
as.numeric(format(d[1],"%m"))))
    #try(plot(decompose(data.ts)))
    #acf(data.ts)
    try(data.ets <- ets(data.ts))
    try(forecast.ets <- forecast.ets(data.ets, h=24))
    IL <-
c(Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i],Item[i])
    ets.df <- data.frame(forecast.ets)
    ets.df$Item <- IL
    r <- 24*(i-1)+2
    addDataFrame(ets.df, sheet, col.names=FALSE, startRow=r)
  }

  title <- paste("\n\n\n Evaluation of", files[j], " -
Completed at", date(), "\n\n\n")
  cat(title)
  saveWorkbook(outwb, paste(files[j],"xlsx",sep='.'))
}

# close the output file
sink()
dev.off()
#stopCluster(cl)
# change back to the original directory
setwd(initial.dir)


Trevor Miles
Vice President, Thought Leadership
[http://www.kinaxis.com/email-signature/images/logo-kinaxis.png]<http://www.kinaxis.com>
O: +1.613.907.7611  |  M: +1.647.248.6269  |  T:
@MilesAhead<https://twitter.com/milesahead>  |  L:
ca.linkedin.com/in/trevormiles<http://ca.linkedin.com/in/trevormiles>

[Kinexions '14]<http://kinexions.kinaxis.com>

[http://www2.kinaxis.com/email-signature/images/social-icon-twitter.png]<http://twitter.com/kinaxis>
[http://www2.kinaxis.com/email-signature/images/social-icon-facebook.png]
<http://www.facebook.com/Kinaxis>  
[http://www2.kinaxis.com/email-signature/images/social-icon-linkedin.png]
<http://www.linkedin.com/company/kinaxis>  
[http://www2.kinaxis.com/email-signature/images/social-icon-community.png]
<https://community.kinaxis.com>

Confidential. This email and any attachments hereto may contain private,
confidential, and privileged material for the sole use of the addressee. Any
review, copying, or distribution of this email (or any attachments thereto) by
others is strictly prohibited. If you are not the intended recipient, please
return this email to the sender immediately and permanently delete the original
and any copies of this email and any of its attachments. Thank you.


	[[alternative HTML version deleted]]