R help - Jun 2015 - how to change the "ff" properties of a "ff"-related R object after the original "ff" output folder has been moved

Hi all,

I'm new to "ff" package through the using Bioconductor package
"crlmm".  Here is my problem:

I've created a few R objects (e.g. an CNSet) using crlmm based on my data
and save them in a .RData file.  crlmm heavily uses ff package to store results
on a local folder.  For certain reasons, I have moved the ff output folder to
somewhere else.  Now when I go back to R, I can't open those CNSet, for
example, anymore, as the file has a property still storing the old ff output
folder path.

My question is: is there a quick way to change these paths to the new one, so I
don't have to re-run the own analysis.

Many thanks!

Tao

On Jun 25, 2015, at 4:04 PM, Shi, Tao via R-help wrote:
> Hi all,
> 
> I'm new to "ff" package through the using Bioconductor
package "crlmm".  Here is my problem:
> 
> I've created a few R objects (e.g. an CNSet) using crlmm based on my
data and save them in a .RData file.  crlmm heavily uses ff package to store
results on a local folder.  For certain reasons, I have moved the ff output
folder to somewhere else.  Now when I go back to R, I can't open those
CNSet, for example, anymore, as the file has a property still storing the old ff
output folder path.
> 
> My question is: is there a quick way to change these paths to the new one,
so I don't have to re-run the own analysis.
> 
The way to approach this is to create a small example that illustrates teh
problem. In many case all will becove clear, but if not then you will have
something that you can then post ... IN R CODE ... that will let us (most of
whom are not ff-urers) see the problem.

--
David Winsemius
Alameda, CA, USA

Tao,

I do assume that the ff-files are still at some location and not deleted 
by a finalizer. The following explains how to manipulate file locations 
with ff and ffdf objects.

Kind regards
jens

library(ff)
path1 <- "c:/tmp"
path2 <- "c:/tmp2"
# create ffdf,
# using non-standard path sets finalizer to 'close' instead of
'delete'
fdf1 <- as.ffdf(iris,
col_args=list(pattern=file.path(path1,"iris")))
# let's copy the old metadata (but not the files, useclone for that)
# using ffs hybrid copying semantics
fdf2 <- fdf1
# note both are open
is.open(fdf1)
is.open(fdf2)
# close the files
close(fdf1)
# and note that
is.open(fdf1)
is.open(fdf2)
# the magic has kept physical metadata in synch even in the copy
# (virtual metadata is not kept in synch
# which allows different virtual views into the same files
# not unlike SQL VIEWs virtualize dastabase TABLEs)

# filename on a ffdf
filename(fdf2)
# is a shortcut for
lapply(physical(fdf2), filename)
# so filename is a physical attribute
# actually moving the files can be done with the filename<- method
lapply(physical(fdf2), function(x)filename(x) <- sub(path1, path2, 
filename(x)))
# check this
filename(fdf1)
filename(fdf2)

# filename on ff
filename(fdf1$Species)
# is a shortcut for
attr(attr(fdf1$Species, "physical"), "filename")
# and if you directly manipulate this attribute
# you circummvent the filename method
# and the file itself will not be moved
attr(attr(fdf1$Species, "physical"), "filename") <-
sub(path2, path1,
filename(fdf1$Species))
# now the metadata points to a different location
filename(fdf1$Species)
# note that this physical attribute was also changed
# for the copy
filename(fdf2$Species)
# of course you can fix the erroneous metadata by
attr(attr(fdf1$Species, "physical"), "filename") <-
sub(path1, path2,
filename(fdf1$Species))
# or for all columns in a ffdf by
lapply(physical(fdf2), function(x)attr(attr(x, "physical"),
"filename")
<- sub(path2, path1, filename(x)))
# now we have your situation with broken metadata
open(fdf2)
# and can fix that by
lapply(physical(fdf2), function(x)attr(attr(x, "physical"),
"filename")
<- sub(path1, path2, filename(x)))
# check
open(fdf2)



Am 26.06.2015 um 01:04 schrieb Shi, Tao:
> Hi all,
>
> I'm new to "ff" package through the using Bioconductor
package "crlmm".  Here is my problem:
>
> I've created a few R objects (e.g. an CNSet) using crlmm based on my
data and save them in a .RData file.  crlmm heavily uses ff package to store
results on a local folder.  For certain reasons, I have moved the ff output
folder to somewhere else.  Now when I go back to R, I can't open those
CNSet, for example, anymore, as the file has a property still storing the old ff
output folder path.
>
> My question is: is there a quick way to change these paths to the new one,
so I don't have to re-run the own analysis.
>
> Many thanks!
>
> Tao
>

Hi David,
Normally, I would have a R code example for this, but since I'm new to this
too and just didn't have time to do that. ?Hopefully, Jen's code
illustrates the problem.
Tao
 


     On Thursday, June 25, 2015 8:53 PM, David Winsemius <dwinsemius at
comcast.net> wrote:
   

 
On Jun 25, 2015, at 4:04 PM, Shi, Tao via R-help wrote:
> Hi all,
> 
> I'm new to "ff" package through the using Bioconductor
package "crlmm".? Here is my problem:
> 
> I've created a few R objects (e.g. an CNSet) using crlmm based on my
data and save them in a .RData file.? crlmm heavily uses ff package to store
results on a local folder.? For certain reasons, I have moved the ff output
folder to somewhere else.? Now when I go back to R, I can't open those
CNSet, for example, anymore, as the file has a property still storing the old ff
output folder path.
> 
> My question is: is there a quick way to change these paths to the new one,
so I don't have to re-run the own analysis.
> 
The way to approach this is to create a small example that illustrates teh
problem. In many case all will becove clear, but if not then you will have
something that you can then post ... IN R CODE ... that will let us (most of
whom are not ff-urers) see the problem.

--
David Winsemius
Alameda, CA, USA


  
	[[alternative HTML version deleted]]

Hi Jens,
Thanks for the example! ?I can see that you can change the 'filename'
attribute for a ff object one-by-one, but is there a way to issue one command
that will automatically change the attribute to all the ff objects in your
workspace, as you can imagine when I have large number of objects like this,
doing one-by-one is cumbersome? ?After all, all it needs to change is the folder
path, right?
May be this is a too wishful thinking :-)
Best,
Tao
 


     On Friday, June 26, 2015 3:31 AM, Jens Oehlschl?gel <Jens.Oehlschlaegel
at truecluster.com> wrote:
   

 Tao,

I do assume that the ff-files are still at some location and not deleted 
by a finalizer. The following explains how to manipulate file locations 
with ff and ffdf objects.

Kind regards
jens

library(ff)
path1 <- "c:/tmp"
path2 <- "c:/tmp2"
# create ffdf,
# using non-standard path sets finalizer to 'close' instead of
'delete'
fdf1 <- as.ffdf(iris,
col_args=list(pattern=file.path(path1,"iris")))
# let's copy the old metadata (but not the files, useclone for that)
# using ffs hybrid copying semantics
fdf2 <- fdf1
# note both are open
is.open(fdf1)
is.open(fdf2)
# close the files
close(fdf1)
# and note that
is.open(fdf1)
is.open(fdf2)
# the magic has kept physical metadata in synch even in the copy
# (virtual metadata is not kept in synch
# which allows different virtual views into the same files
# not unlike SQL VIEWs virtualize dastabase TABLEs)

# filename on a ffdf
filename(fdf2)
# is a shortcut for
lapply(physical(fdf2), filename)
# so filename is a physical attribute
# actually moving the files can be done with the filename<- method
lapply(physical(fdf2), function(x)filename(x) <- sub(path1, path2, 
filename(x)))
# check this
filename(fdf1)
filename(fdf2)

# filename on ff
filename(fdf1$Species)
# is a shortcut for
attr(attr(fdf1$Species, "physical"), "filename")
# and if you directly manipulate this attribute
# you circummvent the filename method
# and the file itself will not be moved
attr(attr(fdf1$Species, "physical"), "filename") <-
sub(path2, path1,
filename(fdf1$Species))
# now the metadata points to a different location
filename(fdf1$Species)
# note that this physical attribute was also changed
# for the copy
filename(fdf2$Species)
# of course you can fix the erroneous metadata by
attr(attr(fdf1$Species, "physical"), "filename") <-
sub(path1, path2,
filename(fdf1$Species))
# or for all columns in a ffdf by
lapply(physical(fdf2), function(x)attr(attr(x, "physical"),
"filename")
<- sub(path2, path1, filename(x)))
# now we have your situation with broken metadata
open(fdf2)
# and can fix that by
lapply(physical(fdf2), function(x)attr(attr(x, "physical"),
"filename")
<- sub(path1, path2, filename(x)))
# check
open(fdf2)



Am 26.06.2015 um 01:04 schrieb Shi, Tao:
> Hi all,
>
> I'm new to "ff" package through the using Bioconductor
package "crlmm".? Here is my problem:
>
> I've created a few R objects (e.g. an CNSet) using crlmm based on my
data and save them in a .RData file.? crlmm heavily uses ff package to store
results on a local folder.? For certain reasons, I have moved the ff output
folder to somewhere else.? Now when I go back to R, I can't open those
CNSet, for example, anymore, as the file has a property still storing the old ff
output folder path.
>
> My question is: is there a quick way to change these paths to the new one,
so I don't have to re-run the own analysis.
>
> Many thanks!
>
> Tao
>

  
	[[alternative HTML version deleted]]