Dear members,
I am using pbmcapply to parellise my code. But the
following code doesn't work:
> LYG <- pbmclapply(LYGH,FUN = arfima,mc.cores = 2,mc.preschedule = FALSE)
|
| 0%, ETA NA^
It just hangs.
But the following works:
> LYG <- pbmclapply(LYGH,FUN = arfima,mc.cores = 2)
|==============================================================================================================================================|
100%, Elapsed 00:32> LYG <- pbmclapply(LYGH,FUN = arfima,mc.cores = 2,mc.preschedule =
FALSE)
|==============================================================================================================================================|
100%, Elapsed 00:52
Any idea on why mc.preschedule = FALSE works after the one without it?
Many thanks ...
THanking you,
Yours sincerely,
AKSHAY M KULKARNI
[[alternative HTML version deleted]]
Dear members,
I think I have a hint:
The doc says:
mc.cleanup
if set to TRUE then all children that have been forked by this
function will be killed (by sending SIGTERM) before this function returns. Under
normal circumstances mclapply waits for the children to deliver results, so this
option usually has only effect when mclapply is interrupted. If set to FALSE
then child processes are collected, but not forcefully terminated. As a special
case this argument can be set to the number of the signal that should be used to
kill the children instead of SIGTERM.
mc.allow.recursive
Unless true, calling mclapply in a child process will use the child and not fork
again.
I think the child processes are not killed. I think I have to use one or both of
the above arguments. Can you please shed some light on which one? ANd what goes
on under the hood?
THanking you,Yours sincerely,AKSHAY M KULKARNI
________________________________
From: R-help <r-help-bounces at r-project.org> on behalf of akshay
kulkarni <akshay_e4 at hotmail.com>
Sent: Friday, June 9, 2023 11:31 PM
To: R help Mailing list <r-help at r-project.org>
Subject: [R] inconsistency in mclapply.....
Dear members,
I am using pbmcapply to parellise my code. But the
following code doesn't work:
> LYG <- pbmclapply(LYGH,FUN = arfima,mc.cores = 2,mc.preschedule = FALSE)
|
| 0%, ETA NA^
It just hangs.
But the following works:
> LYG <- pbmclapply(LYGH,FUN = arfima,mc.cores = 2)
|==============================================================================================================================================|
100%, Elapsed 00:32> LYG <- pbmclapply(LYGH,FUN = arfima,mc.cores = 2,mc.preschedule =
FALSE)
|==============================================================================================================================================|
100%, Elapsed 00:52
Any idea on why mc.preschedule = FALSE works after the one without it?
Many thanks ...
THanking you,
Yours sincerely,
AKSHAY M KULKARNI
[[alternative HTML version deleted]]
______________________________________________
R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]
On Fri, 9 Jun 2023 18:01:44 +0000 akshay kulkarni <akshay_e4 at hotmail.com> wrote:> > LYG <- pbmclapply(LYGH,FUN = arfima,mc.cores = 2,mc.preschedule > > FALSE) > | > | > 0%, ETA NA^ > > It just hangs.My questions from the last time still stand: 0) What is your sessionInfo()? Maybe you're running a parallel BLAS which doesn't always handle fork() or something. It may be worth disabling BLAS-level parallelism as long as you're already trying to use 100% of your processor by other means. 1) What does traceback() show after you interrupt pbmclapply? Most likely, you would be interrupting selectChildren(), but if not, the problem may lie in a very different place from what I'm expecting. 2) While pbmclapply is hung, without interrupting it, take a look at the state of the system and the processes on it (are you still on RHEL? use `top` or whatever task manager you're comfortable with). a) Is 100% of the CPU being used? 100% of one core? Is system mostly idle? b) Can you find the child processes launched by pbmclapply? c) Write down the PID of the child process and attach a debugger to it (If you're on RHEL, try following this guide: <https://beej.us/guide/bggdb/#attach>. If GDB asks you to install additional debug symbols by running debuginfo-install, follow its guidance and then restart GDB.) and obtain a backtrace. (In GDB, the command to obtain a backtrace is "backtrace".) Which function is the child process stuck in? -- Best regards, Ivan