R help - Nov 2012 - problem with package development and older defs earlier in search order

Hi,

I have a problem with a package I have developed in that functions do not get
loaded due to older versions of the functions being in the .GlobalEnv? fetched
from .Rdata files stored from previous saved workspaces. I need to be able to
fix this somehow when I load the package. I do not want to mess up the search
order to fix the problem.

How I got myself into this mess is that I started developing using a bunch of R
files and a dynamic library and then loaded these from an R function rather than
a package. This is great when developing as it allows quick reloading. After
doing this for about a year now I have (unknowingly) populated all my favourite
working directories with .Rdata files that contain various versions of my
functions. And I shared this collection of code, with its flakey loading
mechanism, with PhD students and colleagues who have used it and nicely
populated their working directories with it as well?.

So now I have learnt how to turn it into a package as it is in a more stable
state. But when I load it of course the .GlobalEnv with the definition from old
saved .Rdata files is first in the search order. I say "of course" but
tt has taken me a while to find this out. At first I thought I was going mad as
the old functions seemed to magically turn up even though I had updated the
package and checked that the new versions of the functions were the only ones in
the system (or so I thought)?

My question is: how can I force a package to overwrite the "old"
definitions in the .GlobalEnv. If it cannot be done automatically it could be a
user triggered function. While developing I still want newer versions of
functions to take precedence in the .GlobalEnv over the package versions. i.e. I
do not want the package to be earlier in the search order than .GlobalEnv (if
this is even possible), I just want to be able to delete the old definitions. I
do not want to delete the .Rdata files as there is useful state in them...

Any help getting me out of this mess would be appreciated!

Martin

Suggestions:

(1) Work in a different directory (so that you will have a different --- 
initially
empty) .RData file.

(2) Or:  Clean up the .Rdata file in the directory that you are 
currently using;
rm(list=ls()) does this for you.  Every so slightly dangerous! :-)

(3) Possibly:  Before doing (2), save stuff that you actually *want*, 
e.g. via

     save(mung,gorp,clyde,irving,melvin,file="savedStuff.rda")

Then you can recover mung, gorp, etc. via

     load("savedStuff.rda")

     cheers,

         Rolf Turner

On 08/11/12 13:11, Martin J Reed wrote:
> Hi,
>
> I have a problem with a package I have developed in that functions do not
get loaded due to older versions of the functions being in the .GlobalEnv?
fetched from .Rdata files stored from previous saved workspaces. I need to be
able to fix this somehow when I load the package. I do not want to mess up the
search order to fix the problem.
>
> How I got myself into this mess is that I started developing using a bunch
of R files and a dynamic library and then loaded these from an R function rather
than a package. This is great when developing as it allows quick reloading.
After doing this for about a year now I have (unknowingly) populated all my
favourite working directories with .Rdata files that contain various versions of
my functions. And I shared this collection of code, with its flakey loading
mechanism, with PhD students and colleagues who have used it and nicely
populated their working directories with it as well?.
>
> So now I have learnt how to turn it into a package as it is in a more
stable state. But when I load it of course the .GlobalEnv with the definition
from old saved .Rdata files is first in the search order. I say "of
course" but tt has taken me a while to find this out. At first I thought I
was going mad as the old functions seemed to magically turn up even though I had
updated the package and checked that the new versions of the functions were the
only ones in the system (or so I thought)?
>
> My question is: how can I force a package to overwrite the "old"
definitions in the .GlobalEnv. If it cannot be done automatically it could be a
user triggered function. While developing I still want newer versions of
functions to take precedence in the .GlobalEnv over the package versions. i.e. I
do not want the package to be earlier in the search order than .GlobalEnv (if
this is even possible), I just want to be able to delete the old definitions. I
do not want to delete the .Rdata files as there is useful state in them...
>
> Any help getting me out of this mess would be appreciated!

On 12-11-07 7:11 PM, Martin J Reed wrote:
> Hi,
>
> I have a problem with a package I have developed in that functions do not
get loaded due to older versions of the functions being in the .GlobalEnv?
fetched from .Rdata files stored from previous saved workspaces. I need to be
able to fix this somehow when I load the package. I do not want to mess up the
search order to fix the problem.
>
> How I got myself into this mess is that I started developing using a bunch
of R files and a dynamic library and then loaded these from an R function rather
than a package. This is great when developing as it allows quick reloading.
After doing this for about a year now I have (unknowingly) populated all my
favourite working directories with .Rdata files that contain various versions of
my functions. And I shared this collection of code, with its flakey loading
mechanism, with PhD students and colleagues who have used it and nicely
populated their working directories with it as well?.
>
> So now I have learnt how to turn it into a package as it is in a more
stable state. But when I load it of course the .GlobalEnv with the definition
from old saved .Rdata files is first in the search order. I say "of
course" but tt has taken me a while to find this out. At first I thought I
was going mad as the old functions seemed to magically turn up even though I had
updated the package and checked that the new versions of the functions were the
only ones in the system (or so I thought)?
>
> My question is: how can I force a package to overwrite the "old"
definitions in the .GlobalEnv. If it cannot be done automatically it could be a
user triggered function. While developing I still want newer versions of
functions to take precedence in the .GlobalEnv over the package versions. i.e. I
do not want the package to be earlier in the search order than .GlobalEnv (if
this is even possible), I just want to be able to delete the old definitions. I
do not want to delete the .Rdata files as there is useful state in them...
>
> Any help getting me out of this mess would be appreciated!
Rolf's advice is good:  delete the copies from your global env, and the 
ones in your package won't be hidden.

Here's some more long term advice:  keep your global environment empty 
at the start of each session, and thereby avoid this problem.  If you 
want to temporarily edit a function from your package, you can still 
make a copy in the global environment, and it will take precedence (when 
called from the top level; the package functions that call each other 
won't see it).  At the end of your session, when offered the chance to 
save your workspace, just say no.

Regarding your specific question:  a package could overwrite user 
objects in the global environment, but that's very bad practice.  The 
user owns those, and packages should avoid messing with them.  Let the 
user say

foo <- foo

to make a copy of the foo function in the global environment, don't 
force your own copy there.  (The user will still be able to get back to 
the original one using the pkg::foo notation, and as I mentioned, 
packages will generally look internally first, so they'll see the 
original one.)

Duncan Murdoch

Rolf and Duncan

Many thanks. Your answers pointed me to a refinement that is closer to what I
want:

 
rm(list=intersect(ls(".GlobalEnv"),ls("package:reedgraph")),
     pos=".GlobalEnv")

This only removes items that are "masked" by GlobalEnv from my
package.

As this is a bit long for some of the people that need to update their
workspaces I have created a function to fix it:

<packagename>.update2package  <- function() {
 
rm(list=intersect(ls(".GlobalEnv"),ls("package:<packagename>")),
     pos=".GlobalEnv")
}

Just for completeness (if anyone else reads this). It is possible to make this
happen automatically at package load using

.onAttach <- function(libname, pkgname) {
<packagename>.update2package() }

However, as Duncan says this is REALLY bad practice, but is useful to me while
debugging?.

Duncan: you say avoid saving the workspace. While I can understand why you are
saying this, one of the really powerful features of R is the ability to start R
in a working directory and have the state saved so you can come back to it.
While this may be bad for the GUI versions of R as the workspace would just keep
growing, if you start it from a particular directory this is not a problem and
allows coming back to where you left off?

Regards,

Martin

On 8 Nov 2012, at 02:38, Rolf Turner <rolf.turner at xtra.co.nz> wrote:
> 
> Suggestions:
> 
> (1) Work in a different directory (so that you will have a different ---
initially
> empty) .RData file.
> 
> (2) Or:  Clean up the .Rdata file in the directory that you are currently
using;
> rm(list=ls()) does this for you.  Every so slightly dangerous! :-)
> 
> (3) Possibly:  Before doing (2), save stuff that you actually *want*, e.g.
via
> 
>    save(mung,gorp,clyde,irving,melvin,file="savedStuff.rda")
> 
> Then you can recover mung, gorp, etc. via
> 
>    load("savedStuff.rda")
> 
>    cheers,
> 
>        Rolf Turner
> 
> On 08/11/12 13:11, Martin J Reed wrote:
>> Hi,
>> 
>> I have a problem with a package I have developed in that functions do
not get loaded due to older versions of the functions being in the .GlobalEnv?
fetched from .Rdata files stored from previous saved workspaces. I need to be
able to fix this somehow when I load the package. I do not want to mess up the
search order to fix the problem.
>> 
>> How I got myself into this mess is that I started developing using a
bunch of R files and a dynamic library and then loaded these from an R function
rather than a package. This is great when developing as it allows quick
reloading. After doing this for about a year now I have (unknowingly) populated
all my favourite working directories with .Rdata files that contain various
versions of my functions. And I shared this collection of code, with its flakey
loading mechanism, with PhD students and colleagues who have used it and nicely
populated their working directories with it as well?.
>> 
>> So now I have learnt how to turn it into a package as it is in a more
stable state. But when I load it of course the .GlobalEnv with the definition
from old saved .Rdata files is first in the search order. I say "of
course" but tt has taken me a while to find this out. At first I thought I
was going mad as the old functions seemed to magically turn up even though I had
updated the package and checked that the new versions of the functions were the
only ones in the system (or so I thought)?
>> 
>> My question is: how can I force a package to overwrite the
"old" definitions in the .GlobalEnv. If it cannot be done
automatically it could be a user triggered function. While developing I still
want newer versions of functions to take precedence in the .GlobalEnv over the
package versions. i.e. I do not want the package to be earlier in the search
order than .GlobalEnv (if this is even possible), I just want to be able to
delete the old definitions. I do not want to delete the .Rdata files as there is
useful state in them...
>> 
>> Any help getting me out of this mess would be appreciated!