Shi, Tao
2015-Jun-25 23:04 UTC
[R] how to change the "ff" properties of a "ff"-related R object after the original "ff" output folder has been moved
Hi all, I'm new to "ff" package through the using Bioconductor package "crlmm". Here is my problem: I've created a few R objects (e.g. an CNSet) using crlmm based on my data and save them in a .RData file. crlmm heavily uses ff package to store results on a local folder. For certain reasons, I have moved the ff output folder to somewhere else. Now when I go back to R, I can't open those CNSet, for example, anymore, as the file has a property still storing the old ff output folder path. My question is: is there a quick way to change these paths to the new one, so I don't have to re-run the own analysis. Many thanks! Tao
David Winsemius
2015-Jun-26 03:53 UTC
[R] how to change the "ff" properties of a "ff"-related R object after the original "ff" output folder has been moved
On Jun 25, 2015, at 4:04 PM, Shi, Tao via R-help wrote:> Hi all, > > I'm new to "ff" package through the using Bioconductor package "crlmm". Here is my problem: > > I've created a few R objects (e.g. an CNSet) using crlmm based on my data and save them in a .RData file. crlmm heavily uses ff package to store results on a local folder. For certain reasons, I have moved the ff output folder to somewhere else. Now when I go back to R, I can't open those CNSet, for example, anymore, as the file has a property still storing the old ff output folder path. > > My question is: is there a quick way to change these paths to the new one, so I don't have to re-run the own analysis. >The way to approach this is to create a small example that illustrates teh problem. In many case all will becove clear, but if not then you will have something that you can then post ... IN R CODE ... that will let us (most of whom are not ff-urers) see the problem. -- David Winsemius Alameda, CA, USA
Jens Oehlschlägel
2015-Jun-26 10:31 UTC
[R] how to change the "ff" properties of a "ff"-related R object after the original "ff" output folder has been moved
Tao, I do assume that the ff-files are still at some location and not deleted by a finalizer. The following explains how to manipulate file locations with ff and ffdf objects. Kind regards jens library(ff) path1 <- "c:/tmp" path2 <- "c:/tmp2" # create ffdf, # using non-standard path sets finalizer to 'close' instead of 'delete' fdf1 <- as.ffdf(iris, col_args=list(pattern=file.path(path1,"iris"))) # let's copy the old metadata (but not the files, useclone for that) # using ffs hybrid copying semantics fdf2 <- fdf1 # note both are open is.open(fdf1) is.open(fdf2) # close the files close(fdf1) # and note that is.open(fdf1) is.open(fdf2) # the magic has kept physical metadata in synch even in the copy # (virtual metadata is not kept in synch # which allows different virtual views into the same files # not unlike SQL VIEWs virtualize dastabase TABLEs) # filename on a ffdf filename(fdf2) # is a shortcut for lapply(physical(fdf2), filename) # so filename is a physical attribute # actually moving the files can be done with the filename<- method lapply(physical(fdf2), function(x)filename(x) <- sub(path1, path2, filename(x))) # check this filename(fdf1) filename(fdf2) # filename on ff filename(fdf1$Species) # is a shortcut for attr(attr(fdf1$Species, "physical"), "filename") # and if you directly manipulate this attribute # you circummvent the filename method # and the file itself will not be moved attr(attr(fdf1$Species, "physical"), "filename") <- sub(path2, path1, filename(fdf1$Species)) # now the metadata points to a different location filename(fdf1$Species) # note that this physical attribute was also changed # for the copy filename(fdf2$Species) # of course you can fix the erroneous metadata by attr(attr(fdf1$Species, "physical"), "filename") <- sub(path1, path2, filename(fdf1$Species)) # or for all columns in a ffdf by lapply(physical(fdf2), function(x)attr(attr(x, "physical"), "filename") <- sub(path2, path1, filename(x))) # now we have your situation with broken metadata open(fdf2) # and can fix that by lapply(physical(fdf2), function(x)attr(attr(x, "physical"), "filename") <- sub(path1, path2, filename(x))) # check open(fdf2) Am 26.06.2015 um 01:04 schrieb Shi, Tao:> Hi all, > > I'm new to "ff" package through the using Bioconductor package "crlmm". Here is my problem: > > I've created a few R objects (e.g. an CNSet) using crlmm based on my data and save them in a .RData file. crlmm heavily uses ff package to store results on a local folder. For certain reasons, I have moved the ff output folder to somewhere else. Now when I go back to R, I can't open those CNSet, for example, anymore, as the file has a property still storing the old ff output folder path. > > My question is: is there a quick way to change these paths to the new one, so I don't have to re-run the own analysis. > > Many thanks! > > Tao >
Shi, Tao
2015-Jun-26 17:55 UTC
[R] how to change the "ff" properties of a "ff"-related R object after the original "ff" output folder has been moved
Hi David, Normally, I would have a R code example for this, but since I'm new to this too and just didn't have time to do that. ?Hopefully, Jen's code illustrates the problem. Tao On Thursday, June 25, 2015 8:53 PM, David Winsemius <dwinsemius at comcast.net> wrote: On Jun 25, 2015, at 4:04 PM, Shi, Tao via R-help wrote:> Hi all, > > I'm new to "ff" package through the using Bioconductor package "crlmm".? Here is my problem: > > I've created a few R objects (e.g. an CNSet) using crlmm based on my data and save them in a .RData file.? crlmm heavily uses ff package to store results on a local folder.? For certain reasons, I have moved the ff output folder to somewhere else.? Now when I go back to R, I can't open those CNSet, for example, anymore, as the file has a property still storing the old ff output folder path. > > My question is: is there a quick way to change these paths to the new one, so I don't have to re-run the own analysis. >The way to approach this is to create a small example that illustrates teh problem. In many case all will becove clear, but if not then you will have something that you can then post ... IN R CODE ... that will let us (most of whom are not ff-urers) see the problem. -- David Winsemius Alameda, CA, USA [[alternative HTML version deleted]]
Shi, Tao
2015-Jun-26 18:01 UTC
[R] how to change the "ff" properties of a "ff"-related R object after the original "ff" output folder has been moved
Hi Jens, Thanks for the example! ?I can see that you can change the 'filename' attribute for a ff object one-by-one, but is there a way to issue one command that will automatically change the attribute to all the ff objects in your workspace, as you can imagine when I have large number of objects like this, doing one-by-one is cumbersome? ?After all, all it needs to change is the folder path, right? May be this is a too wishful thinking :-) Best, Tao On Friday, June 26, 2015 3:31 AM, Jens Oehlschl?gel <Jens.Oehlschlaegel at truecluster.com> wrote: Tao, I do assume that the ff-files are still at some location and not deleted by a finalizer. The following explains how to manipulate file locations with ff and ffdf objects. Kind regards jens library(ff) path1 <- "c:/tmp" path2 <- "c:/tmp2" # create ffdf, # using non-standard path sets finalizer to 'close' instead of 'delete' fdf1 <- as.ffdf(iris, col_args=list(pattern=file.path(path1,"iris"))) # let's copy the old metadata (but not the files, useclone for that) # using ffs hybrid copying semantics fdf2 <- fdf1 # note both are open is.open(fdf1) is.open(fdf2) # close the files close(fdf1) # and note that is.open(fdf1) is.open(fdf2) # the magic has kept physical metadata in synch even in the copy # (virtual metadata is not kept in synch # which allows different virtual views into the same files # not unlike SQL VIEWs virtualize dastabase TABLEs) # filename on a ffdf filename(fdf2) # is a shortcut for lapply(physical(fdf2), filename) # so filename is a physical attribute # actually moving the files can be done with the filename<- method lapply(physical(fdf2), function(x)filename(x) <- sub(path1, path2, filename(x))) # check this filename(fdf1) filename(fdf2) # filename on ff filename(fdf1$Species) # is a shortcut for attr(attr(fdf1$Species, "physical"), "filename") # and if you directly manipulate this attribute # you circummvent the filename method # and the file itself will not be moved attr(attr(fdf1$Species, "physical"), "filename") <- sub(path2, path1, filename(fdf1$Species)) # now the metadata points to a different location filename(fdf1$Species) # note that this physical attribute was also changed # for the copy filename(fdf2$Species) # of course you can fix the erroneous metadata by attr(attr(fdf1$Species, "physical"), "filename") <- sub(path1, path2, filename(fdf1$Species)) # or for all columns in a ffdf by lapply(physical(fdf2), function(x)attr(attr(x, "physical"), "filename") <- sub(path2, path1, filename(x))) # now we have your situation with broken metadata open(fdf2) # and can fix that by lapply(physical(fdf2), function(x)attr(attr(x, "physical"), "filename") <- sub(path1, path2, filename(x))) # check open(fdf2) Am 26.06.2015 um 01:04 schrieb Shi, Tao:> Hi all, > > I'm new to "ff" package through the using Bioconductor package "crlmm".? Here is my problem: > > I've created a few R objects (e.g. an CNSet) using crlmm based on my data and save them in a .RData file.? crlmm heavily uses ff package to store results on a local folder.? For certain reasons, I have moved the ff output folder to somewhere else.? Now when I go back to R, I can't open those CNSet, for example, anymore, as the file has a property still storing the old ff output folder path. > > My question is: is there a quick way to change these paths to the new one, so I don't have to re-run the own analysis. > > Many thanks! > > Tao >[[alternative HTML version deleted]]