R help - Feb 2017 - How to do double (nested) parSapply in R?

Hello,

I have a data orig.raw that contains 8 predictors and 1 outcome variable.
I'm trying to run simulation (bootstrap) and create, let's say, 10
confidence intervals for coefficients estimated by LASSO. I did it with
regular function replicate, but now I want to do it by using parallel
programming. Here is my code:

cl <- makeCluster(detectCores())
clusterEvalQ(cl,library(glmnet))
clusterEvalQ(cl,library(MASS))
clusterExport(cl,c("orig.raw"))

pp=parSapply(cl,1:10,function(i,data=orig.raw,...){
  library(parallel)
  cl <- makeCluster(detectCores())
  clusterEvalQ(cl,library(glmnet))
  clusterEvalQ(cl,library(MASS))
  clusterExport(cl,c("orig.raw"))

  repl=parSapply(cl=cl,1:10,function(i,data=orig.raw,...){
    s1=data[sample(nrow(data),size=500,replace=TRUE),]
    cross=cv.glmnet(s1[,1:8],s1[,9])
    penalty <- cross$lambda.min
    fit=glmnet(s1[,1:8],s1[,9],alpha=1,lambda=penalty)
    coe=coef(fit)
    m=as(coe, "matrix")
    return(m)
  })

stopCluster(cl)

  mr=t(matrix(repl,nrow = 9,ncol=10))
  means=colMeans(mr)
  std=apply(mr, 2, sd)
  lb=means-1.96*std;
  ub=means+1.96*std;
  ind=t(as.numeric({beta>lb & beta<ub}))
  return(ind)})
stopCluster(cl)

And here is the error I'm getting

Error in checkForRemoteErrors(val) : 8 nodes produced errors; first
error: comparison (6) is possible only for atomic and list types

If I run only function repl - it works and I get the matrix that contains
coefficients from 10 runs.

Can you please help me to solve the problem?
Regards,
Ariel

	[[alternative HTML version deleted]]

Quick comment: sapply() / parSapply() can behaves "unexpectedly". To
troubleshoot this, use parLapply() instead to see if you at least get the
individual results you think you should get.

Henrik

On Feb 2, 2017 08:03, "Art U" <art.tem.us at gmail.com> wrote:
> Hello,
>
> I have a data orig.raw that contains 8 predictors and 1 outcome variable.
> I'm trying to run simulation (bootstrap) and create, let's say, 10
> confidence intervals for coefficients estimated by LASSO. I did it with
> regular function replicate, but now I want to do it by using parallel
> programming. Here is my code:
>
> cl <- makeCluster(detectCores())
> clusterEvalQ(cl,library(glmnet))
> clusterEvalQ(cl,library(MASS))
> clusterExport(cl,c("orig.raw"))
>
> pp=parSapply(cl,1:10,function(i,data=orig.raw,...){
>   library(parallel)
>   cl <- makeCluster(detectCores())
>   clusterEvalQ(cl,library(glmnet))
>   clusterEvalQ(cl,library(MASS))
>   clusterExport(cl,c("orig.raw"))
>
>   repl=parSapply(cl=cl,1:10,function(i,data=orig.raw,...){
>     s1=data[sample(nrow(data),size=500,replace=TRUE),]
>     cross=cv.glmnet(s1[,1:8],s1[,9])
>     penalty <- cross$lambda.min
>     fit=glmnet(s1[,1:8],s1[,9],alpha=1,lambda=penalty)
>     coe=coef(fit)
>     m=as(coe, "matrix")
>     return(m)
>   })
>
> stopCluster(cl)
>
>   mr=t(matrix(repl,nrow = 9,ncol=10))
>   means=colMeans(mr)
>   std=apply(mr, 2, sd)
>   lb=means-1.96*std;
>   ub=means+1.96*std;
>   ind=t(as.numeric({beta>lb & beta<ub}))
>   return(ind)})
> stopCluster(cl)
>
> And here is the error I'm getting
>
> Error in checkForRemoteErrors(val) : 8 nodes produced errors; first
> error: comparison (6) is possible only for atomic and list types
>
> If I run only function repl - it works and I get the matrix that contains
> coefficients from 10 runs.
>
> Can you please help me to solve the problem?
> Regards,
> Ariel
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
	[[alternative HTML version deleted]]