R devel - Jan 2017 - parallel::mc*: Is it possible for a child process to know it is a fork?

When using multicore-forking of the parallel package, is it possible
for a child process to know that it is a fork?  Something like:

  parallel::mclapply(1:10, FUN = function(i) { test_if_running_in_a_fork() })

I'm looking into ways to protect against further parallel processes
(including threads), which not necessarily are created via the
parallel:mc* API, are being spawned off recursively.  For instance,
there are several packages that by default perform multi-threaded
processing using native code, but I'm not sure there's a way for such
package to avoid running in multi-threaded mode if running in a forked
child R processes.  Imagine

  y <- parallel::mclapply(1:10, FUN = function(i) {
     somepkg::threaded_calculation_using_all_cores()
  })

where the developer of `somepkg` is off no control whether user calls
it via mclapply() or via lapply().   I can see how the user of
mclapply() / lapply() can pass on this information, but that's not
safe and it might not be that the user is aware that deep down in the
dependency hierarchy there's one or more functions that do
multi-thread/process processing.

Thanks,

Henrik

On Tue, Jan 24, 2017 at 7:06 PM, Henrik Bengtsson
<henrik.bengtsson at gmail.com> wrote:
> When using multicore-forking of the parallel package, is it possible
> for a child process to know that it is a fork?
R internally uses R_isForkedChild to prevent certain operations within
the fork. However I don't think this is exported anywhere. You could
do something like:

  extern Rboolean R_isForkedChild;
  SEXP is_forked(){
    return ScalarLogical(R_isForkedChild);
  }

But that won't be allowed on CRAN:

* checking compiled code ... NOTE
  Found non-API call to R: ?R_isForkedChild?
  Compiled code should not call non-API entry points in R.

Another method would be to look at getppid(2) and getpgid(2) to lookup
the parent-id and group-id of the current process and test if it
matches that of the (parent) R process.

If you are only interested in limiting further parallelization within
the fork, perhaps you can simply use parallel::mcaffinity to restrict
the forked process to a single core.

On Tue, Jan 24, 2017 at 8:10 PM, Jeroen Ooms <jeroenooms at gmail.com>
wrote:
> On Tue, Jan 24, 2017 at 7:06 PM, Henrik Bengtsson
> <henrik.bengtsson at gmail.com> wrote:
>> When using multicore-forking of the parallel package, is it possible
>> for a child process to know that it is a fork?
>
> R internally uses R_isForkedChild to prevent certain operations within
> the fork. However I don't think this is exported anywhere. You could
> do something like:
>
>   extern Rboolean R_isForkedChild;
>   SEXP is_forked(){
>     return ScalarLogical(R_isForkedChild);
>   }
>
> But that won't be allowed on CRAN:
>
> * checking compiled code ... NOTE
>   Found non-API call to R: ?R_isForkedChild?
>   Compiled code should not call non-API entry points in R.
Yes, that's a bummer.  It could be useful to have this exposed.  It's
used by several core packages, not just 'parallel' itself;

$ grep -F R_isForkedChild -r --include="*.h"
src/include/Defn.h:extern Rboolean R_isForkedChild INI_as(FALSE); /*
was this forked? */

$ grep -F R_isForkedChild -r --include="*.c"
src/library/tcltk/src/tcltk_unix.c://extern Rboolean R_isForkedChild;
src/library/tcltk/src/tcltk_unix.c:    if (!R_isForkedChild && !Tcl_lock
src/library/parallel/src/fork.c:#include <Defn.h> // for R_isForkedChild
src/library/parallel/src/fork.c: R_isForkedChild = 1;
src/modules/X11/devX11.c:    while (!R_isForkedChild && displayOpen
&&
XPending(display)) {
src/modules/X11/devX11.c:    if(R_isForkedChild)
src/unix/sys-unix.c:    if (ptr_R_ProcessEvents && !R_isForkedChild)
ptr_R_ProcessEvents();
>
> Another method would be to look at getppid(2) and getpgid(2) to lookup
> the parent-id and group-id of the current process and test if it
> matches that of the (parent) R process.
I'm not 100% sure I follow.  Is the idea similar to the following in R?

ppid <- Sys.getpid()
is_child <- parallel::mclapply(1:10, FUN = function(i) { Sys.getpid() != ppid
})

How can the child process know 'ppid'?  getppid would give the parent
PID for any process, which could be a non-R process.
>
> If you are only interested in limiting further parallelization within
> the fork, perhaps you can simply use parallel::mcaffinity to restrict
> the forked process to a single core.
This is tied to parallelization via parallel::mc*, correct?  That is,
is it only parallel:::mcfork() that respects those settings or does
this go down deeper in the OS such that it affects forking / threading
on a more general level?

Thanks for your pointers and suggestions,

Henrik