search for: sendmast

...TRUE. Its structure is printed. All outputs are created successfully. So far so good. But then "part i written successfully." it is printed another X times, for values of i for which it was already printed. This output is intermingled with X-1 times the following error message: Error in sendMaster(try(eval(expr, env), silent = TRUE)) : write error, closing pipe to the master Calls: lapply ... <Anonymous> -> <Anonymous> -> mcparallel -> sendMaster and Y times the message "Execution halted". mc_ret is printed again, now being a list of length 85, with the fi...

Hi again, One important correction of my first message: I misinterpreted the output. Actually in that R session 2 input files were processed one after the other in a loop. The first (with 88 parts went fine). The second (with 85 parts) produced the sendMaster errors and failed. If (in a new session via Rscript) I only process the second input file it will work. The other observations on R vs Rscript, NFS share etc. still hold. Sorry for this! Regards, Andreas 2019-11-27 12:10 GMT+01:00 Andreas Kersting<r-devel at akersting.de>: > Hi, >...

...9 12:40 PM, Andreas Kersting wrote: > Hi again, > > One important correction of my first message: I misinterpreted the output. Actually in that R session 2 input files were processed one after the other in a loop. The first (with 88 parts went fine). The second (with 85 parts) produced the sendMaster errors and failed. If (in a new session via Rscript) I only process the second input file it will work. The other observations on R vs Rscript, NFS share etc. still hold. > > Sorry for this! Regards, > Andreas > > 2019-11-27 12:10 GMT+01:00 Andreas Kersting<r-devel at akersting....

...Andreas Kersting wrote: >> Hi again, >> >> One important correction of my first message: I misinterpreted the output. Actually in that R session 2 input files were processed one after the other in a loop. The first (with 88 parts went fine). The second (with 85 parts) produced the sendMaster errors and failed. If (in a new session via Rscript) I only process the second input file it will work. The other observations on R vs Rscript, NFS share etc. still hold. >> >> Sorry for this! Regards, >> Andreas >> >> 2019-11-27 12:10 GMT+01:00 Andreas Kersting<r...

...ting wrote: >>> Hi again, >>> >>> One important correction of my first message: I misinterpreted the output. Actually in that R session 2 input files were processed one after the other in a loop. The first (with 88 parts went fine). The second (with 85 parts) produced the sendMaster errors and failed. If (in a new session via Rscript) I only process the second input file it will work. The other observations on R vs Rscript, NFS share etc. still hold. >>> >>> Sorry for this! Regards, >>> Andreas >>> >>> 2019-11-27 12:10 GMT+01:00 A...

...nitialized variable (reproducibly) was holding the value 7, which corresponded to the file descriptor of the write end of the pipe the second child would use to send its results to the master. Hence, the child unintentionally closed this pipe directly after fork in close_fds_child_ci() resulting in sendMaster() later failing with EBADF. It was fixed with this commit: https://github.com/wch/r-source/commit/e08cffac1c5b9015a1625938d568b648eb1d8aee Regards, Andreas 2019-11-28 13:54 GMT+01:00 Andreas Kersting<r-devel at akersting.de>: > Hi Tomas, > > I rebuild R (v3.5.2 for now, R-devel...

...nitialized variable (reproducibly) was holding the value 7, which corresponded to the file descriptor of the write end of the pipe the second child would use to send its results to the master. Hence, the child unintentionally closed this pipe directly after fork in close_fds_child_ci() resulting in sendMaster() later failing with EBADF. It was fixed with this commit: https://github.com/wch/r-source/commit/e08cffac1c5b9015a1625938d568b648eb1d8aee Regards, Andreas 2019-11-28 13:54 GMT+01:00 Andreas Kersting<r-devel at akersting.de>: > Hi Tomas, > > I rebuild R (v3.5.2 for now, R-devel...

Dear R developers: ... 7: lapply(seq_len(cores), inner.do) 8: FUN(1:3[[3]], ...) 9: sendMaster(try(lapply(X = S, FUN = FUN, ...), silent = TRUE)) Selection: .....................Error in sendMaster(try(lapply(X = S, FUN = FUN, ...), silent = TRUE)) : long vectors not supported yet: memory.c:3100 admittedly, my outcome will be a very big list, with 30,000 elements, each containing data...

...Fedora security settings */ ssize_t n = write(master_fd, &len, sizeof(len)); ... ... } So a pipe write is made in mc_exit, and here's how this function is used in src/library/parallel/R/unix/mcfork.R: mcexit <- function(exit.code = 0L, send = NULL) { if (!is.null(send)) try(sendMaster(send), silent = TRUE) .Call(C_mc_exit, as.integer(exit.code)) } Between sendMaster() and mc_exit() calls, which are made in the child process, the master process may call readChild() followed by rmChild(). rmChild closes the pipe on the master side, and if it's called before child calls...

...hat. The function I'm trying to parallelize is illumqc. I'd like to create a separate process for each of 8 files, contained in the vector "files". Below is my code: for(i in 1:length(files)){ mcparallel(illumqc(files[i]))} I get the following error: Error in sendMaster(serialize(try(eval(expr, env), silent = TRUE), NULL, : ignoring SIGPIPE signal Calls: mcparallel -> sendMaster -> .Call In addition: Warning message: In min(which(alf != 0)) : no non-missing arguments to min; returning Inf However, if I try and make a simpler example, everythin...

...> > ... ... > > } > > > > So a pipe write is made in mc_exit, and here's how this function is > > used in src/library/parallel/R/unix/mcfork.R: > > > > mcexit <- function(exit.code = 0L, send = NULL) > > { > > if (!is.null(send)) try(sendMaster(send), silent = TRUE) > > .Call(C_mc_exit, as.integer(exit.code)) > > } > > > > Between sendMaster() and mc_exit() calls, which are made in the child > > process, the master process may call readChild() followed by > > rmChild(). rmChild closes the pipe o...

...--git a/src/library/parallel/R/unix/mcfork.R b/src/library/parallel/R/unix/mcfork.R index 8e27534..4f92193 100644 --- a/src/library/parallel/R/unix/mcfork.R +++ b/src/library/parallel/R/unix/mcfork.R @@ -82,7 +82,8 @@ mckill <- function(process, signal = 2L) ## used by mcparallel, mclapply sendMaster <- function(what) { - if (!is.raw(what)) what <- serialize(what, NULL, FALSE) + # This is talking to the same machine, so no point in using xdr. + if (!is.raw(what)) what <- serialize(what, NULL, xdr = FALSE) .Call(C_mc_send_master, what, PACKAGE = "parallel")...

...) + nchar(sm[1L], type = "w") if (is.na(w)) 8: w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L], try(lapply(X = S, FUN = FUN, ...), silent = TRUE) type = "b") if (w > LONG) prefix <- paste0(prefix, "\n ") 9: }sendMaster(try(lapply(X = S, FUN = FUN, ...), silent = TRUE)) else prefix <- "Error : " msg <- paste0(prefix, conditionMessage(e), "\n") .Internal(seterrmessage(msg[1L]))10: if (!silent && isTRUE(getOption("show.error.messages"))) {FUN(X[[i]], ...)...

...Fedora security settings */ ssize_t n = write(master_fd, &len, sizeof(len)); ... ... } So a pipe write is made in mc_exit, and here's how this function is used in src/library/parallel/R/unix/mcfork.R: mcexit <- function(exit.code = 0L, send = NULL) { if (!is.null(send)) try(sendMaster(send), silent = TRUE) .Call(C_mc_exit, as.integer(exit.code)) } Between sendMaster() and mc_exit() calls, which are made in the child process, the master process may call readChild() followed by rmChild(). rmChild closes the pipe on the master side, and if it's called before child calls...

...n = write(master_fd, &len, sizeof(len)); > ... ... > } > > So a pipe write is made in mc_exit, and here's how this function is > used in src/library/parallel/R/unix/mcfork.R: > > mcexit <- function(exit.code = 0L, send = NULL) > { > if (!is.null(send)) try(sendMaster(send), silent = TRUE) > .Call(C_mc_exit, as.integer(exit.code)) > } > > Between sendMaster() and mc_exit() calls, which are made in the child > process, the master process may call readChild() followed by > rmChild(). rmChild closes the pipe on the master side, and if it...

...ot;w") > if (is.na(w)) 8: w <- 14L + nchar(dcall, type = "b") + > nchar(sm[1L], try(lapply(X = S, FUN = FUN, ...), silent = TRUE) > type = "b") > if (w > LONG) prefix <- paste0(prefix, "\n ") 9: > }sendMaster(try(lapply(X = S, FUN = FUN, ...), silent = TRUE)) else > prefix <- "Error : " > msg <- paste0(prefix, conditionMessage(e), "\n") > .Internal(seterrmessage(msg[1L]))10: if (!silent && > isTRUE(getOption("show.error.messages")))...

..."w") > if (is.na(w)) 8: w <- 14L + nchar(dcall, type = "b") + > nchar(sm[1L], try(lapply(X = S, FUN = FUN, ...), silent = TRUE) > type = "b") > if (w > LONG) prefix <- paste0(prefix, "\n ") 9: > }sendMaster(try(lapply(X = S, FUN = FUN, ...), silent = TRUE)) else > prefix <- "Error : " > msg <- paste0(prefix, conditionMessage(e), "\n") > .Internal(seterrmessage(msg[1L]))10: if (!silent && > isTRUE(getOption("show.error.messages"))) {...

...eterrmessage(msg[1L])) if (!silent && identical(getOption("show.error.messages"), TRUE)) { cat(msg, file = stderr()) .Internal(printDeferredWarnings()) } invisible(structure(msg, class = "try-error"))}) 18: try(lapply(S, FUN, ...), silent = TRUE) 19: sendMaster(try(lapply(S, FUN, ...), silent = TRUE)) 20: FUN(1:2[[1L]], ...) 21: lapply(1:cores, inner.do) 22: mclapply(argsList, FUN, mc.preschedule = preschedule, mc.set.seed = set.seed, mc.silent = silent, mc.cores = cores) 23: e$fun(obj, substitute(ex), parent.frame(), e$data) 24: foreach(IT = 1:2) %...

I stumbled upon the following: f <- parallel::mcparallel(raw(0L)) parallel::mccollect(f) # $`77083` # NULL but f <- parallel::mcparallel(raw(1L)) parallel::mccollect(f) # Error in unserialize(r) : read error traceback() # 2: unserialize(r) # 1: parallel::mccollect(f) (restarting because the above appears to corrupt the R session) f <- parallel::mcparallel(raw(2L))

Dear all, I have quite large code (with lapply and mclapply) and I am getting the following error. Error: serialization is too large to store in a raw vector Is it possible to ask from R to extend the Error messages with more details? I would like to see where this problem exists. B.R Alex [[alternative HTML version deleted]]