R devel - Jul 2020 - Speed-up/Cache loadNamespace()

Le 20/07/2020 ? 10:15, Abby Spurdle a ?crit?:
> It's possible to run R (or a c parent process) as a background process
> via a named pipe, and then write script files to the named pipe.
> However, the details depend on what shell you use.
>
> The last time I tried (which was a long time ago), I created a small c
> program to run R, read from the named pipe from within c, then wrote
> it's contents to R's standard in.
>
> It might be possible to do it without the c program.
> Haven't checked.
For testing purposes, you can do:

- in a shell 1:
 ?mkfifo rpipe
 ?exec 3>rpipe # without this trick, Rscript will end after the first 
"echo" hereafter or at the end of your first script.

- in a shell 2:
 ?Rscript rfifo

- in a shell 3:
 ?echo "print('hello')" > rpipe
 ?echo "print('hello again')" > rpipe

Then in the shell 2, you will see the output:
[1] "hello"
[1] "hello again"
etc.

If your R scripts contain "stop()" or "q('yes')" or
any other error, it
will end the Rscript process. Kind of watch-dog can be set for automatic 
relaunching if needed. Another way to stop the Rscript process is to 
kill the "exec 3>rpipe" one. You can find its PID with "fuser
rpipe"

Best,
Serguei.
>
>
> On Mon, Jul 20, 2020 at 3:50 AM Mario Annau <mario.annau at
gmail.com> wrote:
>> Dear all,
>>
>> in our current setting we have our packages stored on a (rather slow)
>> network drive and need to invoke short R scripts (using RScript) in a
>> timely manner. Most of the script's runtime is spent with package
loading
>> using library() (or loadNamespace to be precise).
>>
>> Is there a way to cache the package namespaces as listed in
>> loadedNamespaces() and load them into memory before the script is
executed?
>>
>> My first simplistic attempt was to serialize the environment output
>> from loadNamespace() to a file and load it before the script is
started.
>> However, loading the object automatically also loads all the referenced
>> namespaces (from the slow network share) which is undesirable for this
use
>> case.
>>
>> Cheers,
>> Mario
>>
>>          [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

-- 
Serguei Sokol
Ingenieur de recherche INRAE

Cellule math?matiques
TBI, INSA/INRAE UMR 792, INSA/CNRS UMR 5504
135 Avenue de Rangueil
31077 Toulouse Cedex 04

tel: +33 5 61 55 98 49
email: sokol at insa-toulouse.fr
http://www.toulouse-biotechnology-institute.fr/

Thank you Serguei and Gabor.
Great suggestions.
> If your R scripts contain "stop()" or
"q('yes')" or any other error, it
> will end the Rscript process. Kind of watch-dog can be set for automatic
> relaunching if needed.
It should be possible to change the error handling
behavior.
>From within R:
    options (error = function () NULL)

Or something better...

Also, it may be desirable to wipe the global environment (or parts of
it), after each script:

    remove (list = ls (envir=.GlobalEnv, all.names=TRUE) )

Mario, Abby, et al.

Note that there is no fully safe way of unloading packages which register
methods (as answered by Luke Tierney here:
https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16644 ) which makes the
single R session running arbitrary different scripts thing pretty iffy over
the long term. Even swtichr (which tries hard to support something based on
this) only gets "pretty close".

If the scripts are always the same (up to bugfixes, etc) and most
importantly require the same loaded packages then the above won't be an
issue, of course. Just something to be aware of when planning something
like this.

Best,
~G

On Mon, Jul 20, 2020 at 2:59 PM Abby Spurdle <spurdle.a at gmail.com>
wrote:
> Thank you Serguei and Gabor.
> Great suggestions.
>
> > If your R scripts contain "stop()" or
"q('yes')" or any other error, it
> > will end the Rscript process. Kind of watch-dog can be set for
automatic
> > relaunching if needed.
>
> It should be possible to change the error handling behavior.
> From within R:
>
>     options (error = function () NULL)
>
> Or something better...
>
> Also, it may be desirable to wipe the global environment (or parts of
> it), after each script:
>
>     remove (list = ls (envir=.GlobalEnv, all.names=TRUE) )
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
	[[alternative HTML version deleted]]