R help - May 2017 - loading edited functions already in saved workspace automatically

Am Sat, 6 May 2017 11:17:42 -0400
schrieb Michael Friendly <friendly at yorku.ca>:
> On 5/5/2017 10:23 AM, Ralf Goertz wrote:
> > Am Fri, 05 May 2017 07:14:36 -0700
> > schrieb Jeff Newmiller <jdnewmil at dcn.davis.ca.us>:
> >  
> >> R normally prompts you to save .RData, but it just automatically
> >> saves .Rhistory... the two are unrelated.  
> >
> > Not here. If I say "n" to the prompted question "Save
workspace
> > image? [y/n/c]: " my history doesn't get saved.
> >
> > Version:
> >
> > R version 3.3.1 (2016-06-21) -- "Bug in Your Hair"
> > Copyright (C) 2016 The R Foundation for Statistical Computing
> > Platform: x86_64-suse-linux-gnu (64-bit)
> >  
> 
> On Windoze, here's what I use in my .Rprofile, which runs every time
> I start an RGUI coonsole.  The key is .First & .Last to load/save
> history automagically.
Hi Michael,

thanks. This helps with saving the history without saving the data. But
actually I'd really like to save both and still be able to load
functions automatically from .Rprofile. Not saving the data as Jeff
suggested is not a good option because it is sometimes tedious to
rebuild my environment by reexecuting commands in the history. And I
explained in my OP why I can't use .First() to achieve my goal.

But let me try again to explain the problem because I think not
everybody understood what I was trying to say. For simplicity I use the
plain variable "a" instead of a function. Start a fresh session and
remove all variables, define one variable and quit with saving:
> rm(list=ls())
> a=17
> quit(save="yes")
Now, before opening a new session edit .Rprofile such that it contains
just the two lines:

print("Hello from .Rprofile")
a=42

Start a new session where your saved environment will be loaded.
Observe that you see the line 

[1] "Hello from .Rprofile"

proving that the commands in .Rprofile have been executed. Now look at
"a":
> a
[1] 17


You would expect to see this because *after* your "Hello" line you
find

[Previously saved workspace restored]

So you have set "a" to 42 in .Rprofile but it gets overwritten from
the
previously saved and now restored workspace. On the other hand, .First()
gets executed after the restoring of the workspace. Therefore, I could
edit .Rprofile to read

.First=function(){ assign("a",42,pos=1) }

Now, after starting I see that "a" is indeed 42. But then it turns out
that from now on I need "a" to be 11. After editing .Rprofile
accordingly, I am quite hopeful but after starting a new session I see
that "a" is still 42. Why is that? Because .First() was saved and when
I
started a new session it got a new function body (setting "a" to 11)
but
before it could be executed it was again overwritten by the old value
(setting "a" to 42) and I am chasing my own tail. Sigh.

.Last() doesn't help. Apparently (at least on my linux system) it is
executed *after* saving the environment so too late to remove anything
you don't want saved. In that regard linux doesn't seem to be typical,
since in "?.Last" the reverse order is described as typical:

     Exactly what happens at termination of an R session depends on the
     platform and GUI interface in use.  A typical sequence is to run
     ?.Last()? and ?.Last.sys()? (unless ?runLast? is false), to save
     the workspace if requested (and in most cases also to save the
     session history: see ?savehistory?), then run any finalizers (see
     ?reg.finalizer?) that have been set to be run on exit, close all
     open graphics devices, remove the session temporary directory and
     print any remaining warnings (e.g., from ?.Last()? and device
     closure).


IMHO this is a design flaw.

Ralf

Ralf:

You are afflicted with several mind bugs:
* the "my-way mind bug" -- "I want to do it MY WAY, because
that's sort
of what
I know" and also,
* the "my-square-peg-should-fit-into-this-round-hole mind bug" --
"R
should be able to
do it MY WAY, but it puts obstacles in my path," perhaps a subsidiary, 
but more technical:
* the "loading-a-function-or-data-is-the-same mind bug"
As in many things R, you can't always get to MY WAY from there, at least 
not without a tortuous journey.

You think you should be able to do everything you want in .Rprofile, but 
then you posed two separate problems:
(a) save/reload history
(b) save/reload functions and data

If you recognize them as two separate problems, there is an easier path:
(a) use .Rprofile only for making your history persistent, as I described
(b) Put your functions & data you always want available in a package; 
you can load it from .Rprofile

I originally defined a bunch of handy functions (e.g., cd(), a setwd() 
replacement, that works more like `cd` on unix, in that `cd()` returns 
to the previous directory; it also changes the Windows title to
`RGui:`  abbreviation of getwd() )

I moved them all out of .Rprofile, made a package `myutil` and now load 
them from there with

  #=====================  # load default packages
  #=====================      if (!require(myutil)) warning("myutil
functions not available")

hope this helps,
-Michael

On 5/9/2017 10:20 AM, Ralf Goertz wrote:
> Am Sat, 6 May 2017 11:17:42 -0400
> schrieb Michael Friendly <friendly at yorku.ca>:
>
>> On 5/5/2017 10:23 AM, Ralf Goertz wrote:
>>> Am Fri, 05 May 2017 07:14:36 -0700
>>> schrieb Jeff Newmiller <jdnewmil at dcn.davis.ca.us>:
>>>   
>>>> R normally prompts you to save .RData, but it just
automatically
>>>> saves .Rhistory... the two are unrelated.
>>> Not here. If I say "n" to the prompted question
"Save workspace
>>> image? [y/n/c]: " my history doesn't get saved.
>>>
>>> Version:
>>>
>>> R version 3.3.1 (2016-06-21) -- "Bug in Your Hair"
>>> Copyright (C) 2016 The R Foundation for Statistical Computing
>>> Platform: x86_64-suse-linux-gnu (64-bit)
>>>   
>> On Windoze, here's what I use in my .Rprofile, which runs every
time
>> I start an RGUI coonsole.  The key is .First & .Last to load/save
>> history automagically.
> Hi Michael,
>
> thanks. This helps with saving the history without saving the data. But
> actually I'd really like to save both and still be able to load
> functions automatically from .Rprofile. Not saving the data as Jeff
> suggested is not a good option because it is sometimes tedious to
> rebuild my environment by reexecuting commands in the history. And I
> explained in my OP why I can't use .First() to achieve my goal.
>
> But let me try again to explain the problem because I think not
> everybody understood what I was trying to say. For simplicity I use the
> plain variable "a" instead of a function. Start a fresh session
and
> remove all variables, define one variable and quit with saving:
>
>> rm(list=ls())
>> a=17
>> quit(save="yes")
> Now, before opening a new session edit .Rprofile such that it contains
> just the two lines:
>
> print("Hello from .Rprofile")
> a=42
>
> Start a new session where your saved environment will be loaded.
> Observe that you see the line
>
> [1] "Hello from .Rprofile"
>
> proving that the commands in .Rprofile have been executed. Now look at
> "a":
>
>> a
> [1] 17
>
>
> You would expect to see this because *after* your "Hello" line
you find
>
> [Previously saved workspace restored]
>
> So you have set "a" to 42 in .Rprofile but it gets overwritten
from the
> previously saved and now restored workspace. On the other hand, .First()
> gets executed after the restoring of the workspace. Therefore, I could
> edit .Rprofile to read
>
> .First=function(){ assign("a",42,pos=1) }
>
> Now, after starting I see that "a" is indeed 42. But then it
turns out
> that from now on I need "a" to be 11. After editing .Rprofile
> accordingly, I am quite hopeful but after starting a new session I see
> that "a" is still 42. Why is that? Because .First() was saved and
when I
> started a new session it got a new function body (setting "a" to
11) but
> before it could be executed it was again overwritten by the old value
> (setting "a" to 42) and I am chasing my own tail. Sigh.
>
> .Last() doesn't help. Apparently (at least on my linux system) it is
> executed *after* saving the environment so too late to remove anything
> you don't want saved. In that regard linux doesn't seem to be
typical,
> since in "?.Last" the reverse order is described as typical:
>
>       Exactly what happens at termination of an R session depends on the
>       platform and GUI interface in use.  A typical sequence is to run
>       ?.Last()? and ?.Last.sys()? (unless ?runLast? is false), to save
>       the workspace if requested (and in most cases also to save the
>       session history: see ?savehistory?), then run any finalizers (see
>       ?reg.finalizer?) that have been set to be run on exit, close all
>       open graphics devices, remove the session temporary directory and
>       print any remaining warnings (e.g., from ?.Last()? and device
>       closure).
>
>
> IMHO this is a design flaw.
>
> Ralf
>

-- 
Michael Friendly     Email: friendly AT yorku DOT ca
Professor, Psychology Dept. & Chair, Quantitative Methods
York University      Voice: 416 736-2100 x66249 Fax: 416 736-5814
4700 Keele Street    Web:http://www.datavis.ca
Toronto, ONT  M3J 1P3 CANADA

This boils down to the fact that some "my ways" are more effective in
the long run than others.. but I really want to address the complaint

"... sometimes tedious to rebuild my environment by reexecuting commands in
the history"

by asserting that letting R re-run a script that loads my functions and packages
(though perhaps not the data analysis steps) is always very fast and convenient
to do explicitly. I almost never use the history file feature, because I type
nearly every R instruction I use into a script file and execute/edit it until it
does what I want. I keep functions in a separate file or package, and steps
dealing with a particular data set in their own file that uses source() to load
the functions) even when I am executing the lines interactively. My goal is to
regularly re-execute the whole script so that tomorrow/next year/whenever
someone notices something was wrong then I can re-execute the sequence without
following the dead ends I went down the first time (as using a history file
does) and I don't have a separate clean-up-the-history-file step to go
through to create it. When I have confirmed that the script still works as it
did before then I can find where the analysis/data problem went wrong and fix
it.

This does not mean I never use RData files to reduce how often I re-do slow
calculations... but it does mean that I always have my script that loads the
necessary packages and functions rather than using versions of functions in the
RData file. It is useful to avoid becoming dependent on saved intermediate saved
data files so you don't continue to encounter the effects of script errors
that you have already fixed earlier in the analysis.

It becomes more convenient to minimize dependency on automatic startup behavior
when you want to share your script with someone else or run it yourself on a
different computer (say, a powerful server). If you have the script habit then
these hiccups with moving around are non-issues, and you can perform more and
more complex analyses over time because you don't have to remember all the
individual steps nor do you have to sort through the dead ends in your history
file over and over.

The editor you use can make a huge difference in making this work... get one
that has a hot key that lets you execute one line at a time straight from the
editor rather than requiring an explicit copy/paste. RStudio, Notepad++/NppToR,
IntelliJ IDEA, vim-r, and ESS are a few options I am aware of. RStudio also
supports full screen debugging of R so you can more easily reproduce the exact
conditions where things go wrong inside functions as well.

-- 
Sent from my phone. Please excuse my brevity.

On May 9, 2017 8:49:22 AM PDT, Michael Friendly <friendly at yorku.ca>
wrote:
>Ralf:
>
>You are afflicted with several mind bugs:
>* the "my-way mind bug" -- "I want to do it MY WAY, because
that's sort
>
>of what
>I know" and also,
>* the "my-square-peg-should-fit-into-this-round-hole mind bug" --
"R
>should be able to
>do it MY WAY, but it puts obstacles in my path," perhaps a subsidiary, 
>but more technical:
>* the "loading-a-function-or-data-is-the-same mind bug"
>As in many things R, you can't always get to MY WAY from there, at
>least 
>not without a tortuous journey.
>
>You think you should be able to do everything you want in .Rprofile,
>but 
>then you posed two separate problems:
>(a) save/reload history
>(b) save/reload functions and data
>
>If you recognize them as two separate problems, there is an easier
>path:
>(a) use .Rprofile only for making your history persistent, as I
>described
>(b) Put your functions & data you always want available in a package; 
>you can load it from .Rprofile
>
>I originally defined a bunch of handy functions (e.g., cd(), a setwd() 
>replacement, that works more like `cd` on unix, in that `cd()` returns 
>to the previous directory; it also changes the Windows title to
>`RGui:`  abbreviation of getwd() )
>
>I moved them all out of .Rprofile, made a package `myutil` and now load
>
>them from there with
>
>  #=====================>  # load default packages
>  #=====================>      if (!require(myutil)) warning("myutil
functions not available")
>
>hope this helps,
>-Michael
>
>On 5/9/2017 10:20 AM, Ralf Goertz wrote:
>> Am Sat, 6 May 2017 11:17:42 -0400
>> schrieb Michael Friendly <friendly at yorku.ca>:
>>
>>> On 5/5/2017 10:23 AM, Ralf Goertz wrote:
>>>> Am Fri, 05 May 2017 07:14:36 -0700
>>>> schrieb Jeff Newmiller <jdnewmil at dcn.davis.ca.us>:
>>>>   
>>>>> R normally prompts you to save .RData, but it just
automatically
>>>>> saves .Rhistory... the two are unrelated.
>>>> Not here. If I say "n" to the prompted question
"Save workspace
>>>> image? [y/n/c]: " my history doesn't get saved.
>>>>
>>>> Version:
>>>>
>>>> R version 3.3.1 (2016-06-21) -- "Bug in Your Hair"
>>>> Copyright (C) 2016 The R Foundation for Statistical Computing
>>>> Platform: x86_64-suse-linux-gnu (64-bit)
>>>>   
>>> On Windoze, here's what I use in my .Rprofile, which runs every
time
>>> I start an RGUI coonsole.  The key is .First & .Last to
load/save
>>> history automagically.
>> Hi Michael,
>>
>> thanks. This helps with saving the history without saving the data.
>But
>> actually I'd really like to save both and still be able to load
>> functions automatically from .Rprofile. Not saving the data as Jeff
>> suggested is not a good option because it is sometimes tedious to
>> rebuild my environment by reexecuting commands in the history. And I
>> explained in my OP why I can't use .First() to achieve my goal.
>>
>> But let me try again to explain the problem because I think not
>> everybody understood what I was trying to say. For simplicity I use
>the
>> plain variable "a" instead of a function. Start a fresh
session and
>> remove all variables, define one variable and quit with saving:
>>
>>> rm(list=ls())
>>> a=17
>>> quit(save="yes")
>> Now, before opening a new session edit .Rprofile such that it
>contains
>> just the two lines:
>>
>> print("Hello from .Rprofile")
>> a=42
>>
>> Start a new session where your saved environment will be loaded.
>> Observe that you see the line
>>
>> [1] "Hello from .Rprofile"
>>
>> proving that the commands in .Rprofile have been executed. Now look
>at
>> "a":
>>
>>> a
>> [1] 17
>>
>>
>> You would expect to see this because *after* your "Hello"
line you
>find
>>
>> [Previously saved workspace restored]
>>
>> So you have set "a" to 42 in .Rprofile but it gets
overwritten from
>the
>> previously saved and now restored workspace. On the other hand,
>.First()
>> gets executed after the restoring of the workspace. Therefore, I
>could
>> edit .Rprofile to read
>>
>> .First=function(){ assign("a",42,pos=1) }
>>
>> Now, after starting I see that "a" is indeed 42. But then it
turns
>out
>> that from now on I need "a" to be 11. After editing .Rprofile
>> accordingly, I am quite hopeful but after starting a new session I
>see
>> that "a" is still 42. Why is that? Because .First() was saved
and
>when I
>> started a new session it got a new function body (setting "a"
to 11)
>but
>> before it could be executed it was again overwritten by the old value
>> (setting "a" to 42) and I am chasing my own tail. Sigh.
>>
>> .Last() doesn't help. Apparently (at least on my linux system) it
is
>> executed *after* saving the environment so too late to remove
>anything
>> you don't want saved. In that regard linux doesn't seem to be
>typical,
>> since in "?.Last" the reverse order is described as typical:
>>
>>       Exactly what happens at termination of an R session depends on
>the
>>       platform and GUI interface in use.  A typical sequence is to
>run
>>       ?.Last()? and ?.Last.sys()? (unless ?runLast? is false), to
>save
>>       the workspace if requested (and in most cases also to save the
>>       session history: see ?savehistory?), then run any finalizers
>(see
>>       ?reg.finalizer?) that have been set to be run on exit, close
>all
>>       open graphics devices, remove the session temporary directory
>and
>>       print any remaining warnings (e.g., from ?.Last()? and device
>>       closure).
>>
>>
>> IMHO this is a design flaw.
>>
>> Ralf
>>