R devel - Apr 2020 - Suggestion/opinions: add a `absolute` param to `normalizePath()` to force returning an absolute path

This request stems off a bug report I posted
https://bugs.r-project.org/bugzilla/show_bug.cgi?id=17757 where it was
determined the current behaviour is as expected.

To recap: when given a real file, normalizePath() always* returns the full
absolute path. When given a non-existent file, normalizePath() returns a
full path on Windows but it returns the input on other systems*. I'd argue
that there are benefits to being able to reliably and consistently get a
full path, regardless of whether the file exists or not. In order to not
break existing behaviour, I propose adding an argument `absolute = FALSE`
that will attempt to return an absolute path when the argument is set to
TRUE. I don't have any evidence for this claim, but I believe that others
who use this function would expect, like I did, that an absolute path is
returned regardless of the file state. I understand the documentation is
correct because it warns the absolute path may not be returned, but I
believe it would be a useful feature to support.


* I've tested this on Win7, Win10, two versions of MacOS, ubuntu. This
behaviour may not be true in other OSes

	[[alternative HTML version deleted]]

If we are fiddling with normalizePath, having a way of not following
symlinks (otheer than  ~)  would be useful as well.

I had to write normalizePath2 in switchr for a specific on-the-ground need
to NOT go down all he way to physical paths on a remote compute system
because of how IT handled implementing constant pathing on top of swapping
out hardware, and I can't imagine i'm the only one who has ever faced
such
an issue.

~G

On Tue, Apr 14, 2020 at 10:03 AM Dean Attali <daattali at gmail.com>
wrote:
> This request stems off a bug report I posted
> https://bugs.r-project.org/bugzilla/show_bug.cgi?id=17757 where it was
> determined the current behaviour is as expected.
>
> To recap: when given a real file, normalizePath() always* returns the full
> absolute path. When given a non-existent file, normalizePath() returns a
> full path on Windows but it returns the input on other systems*. I'd
argue
> that there are benefits to being able to reliably and consistently get a
> full path, regardless of whether the file exists or not. In order to not
> break existing behaviour, I propose adding an argument `absolute = FALSE`
> that will attempt to return an absolute path when the argument is set to
> TRUE. I don't have any evidence for this claim, but I believe that
others
> who use this function would expect, like I did, that an absolute path is
> returned regardless of the file state. I understand the documentation is
> correct because it warns the absolute path may not be returned, but I
> believe it would be a useful feature to support.
>
>
> * I've tested this on Win7, Win10, two versions of MacOS, ubuntu. This
> behaviour may not be true in other OSes
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
	[[alternative HTML version deleted]]

I think application-specific normalization is actually the right thing 
to do, because the behavior of normalizePath() is too system-specific 
and too unreliable. That is not a deficiency of R, the same problem 
exists on other systems I've spent too much time recently working on 
normalizePath() implementation on Windows and looking how other systems 
do it, the implementations are very similar, and the documentation often 
less detailed on the limitations.

As the R documentation says, path normalization is meant for displaying 
a user-understandable path name in messages and for comparison (e.g. 
whether two paths refer to the same file, possibly also for their 
canonical location, mapping some permissions to directories). However, 
as it also says, it is only best-effort. The resulting path may not be 
canonical. It may not be pretty (not "user-understandable"). As it may
not be canonical, comparing normalized paths e.g. to decide whether they 
refer to the same file may not be reliable. Again, this is not a 
limitation of R, but a consequence of file-system complexities and also 
differences between platforms. The scope for use of normalizePath() is 
hence very small and only if one can be happy with the "best-effort".

For anything reliable, one should try to use something specific for the 
task, but with well defined behavior. If say just concatenating the 
current working directory and a relative path provides the desired 
behavior in a specific situation, it is best to just do it that way. In 
some cases normalization is easily avoidable by say opening a file 
sooner rather than later (e.g. the use case of Dean in the bug report).

To find out if two path names refer to the same file, with sufficient 
permissions there are better ways both on Unix and Windows (though maybe 
not provided by R base packages).

Disabling symlinks is impossible/infeasible, that would mean 
re-implementing the normalization from scratch inside R, without using 
the OS-level functions (realpath, GetFinalPathNameByHandle), with 
different error behavior on many details, etc, and the result would be 
atypical for both Unix (where realpath is normally used for 
normalization) and Windows platforms (where new language runtimes use 
GetFinalPathNameByHandle, now including R).

Best
Tomas



On 4/14/20 9:51 PM, Gabriel Becker wrote:
> If we are fiddling with normalizePath, having a way of not following
> symlinks (otheer than  ~)  would be useful as well.
>
> I had to write normalizePath2 in switchr for a specific on-the-ground need
> to NOT go down all he way to physical paths on a remote compute system
> because of how IT handled implementing constant pathing on top of swapping
> out hardware, and I can't imagine i'm the only one who has ever
faced such
> an issue.
>
> ~G
>
> On Tue, Apr 14, 2020 at 10:03 AM Dean Attali <daattali at gmail.com>
wrote:
>
>> This request stems off a bug report I posted
>> https://bugs.r-project.org/bugzilla/show_bug.cgi?id=17757 where it was
>> determined the current behaviour is as expected.
>>
>> To recap: when given a real file, normalizePath() always* returns the
full
>> absolute path. When given a non-existent file, normalizePath() returns
a
>> full path on Windows but it returns the input on other systems*.
I'd argue
>> that there are benefits to being able to reliably and consistently get
a
>> full path, regardless of whether the file exists or not. In order to
not
>> break existing behaviour, I propose adding an argument `absolute =
FALSE`
>> that will attempt to return an absolute path when the argument is set
to
>> TRUE. I don't have any evidence for this claim, but I believe that
others
>> who use this function would expect, like I did, that an absolute path
is
>> returned regardless of the file state. I understand the documentation
is
>> correct because it warns the absolute path may not be returned, but I
>> believe it would be a useful feature to support.
>>
>>
>> * I've tested this on Win7, Win10, two versions of MacOS, ubuntu.
This
>> behaviour may not be true in other OSes
>>
>>          [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

The fs[1] function `fs::path_abs()` does what I believe you were
expecting `normalizePath()` to do in this case. e.g.

    setwd("~")
    normalizePath("foo/bar")
    #> Warning in normalizePath("foo/bar") :
    #> path[1]="foo/bar": No such file or directory
    #> [1] "foo/bar"

    fs::path_abs("foo/bar")
    #> /Users/jhester/foo/bar

[1]: https://CRAN.R-project.org/package=fs


On Tue, Apr 14, 2020 at 1:03 PM Dean Attali <daattali at gmail.com>
wrote:
>
> This request stems off a bug report I posted
> https://bugs.r-project.org/bugzilla/show_bug.cgi?id=17757 where it was
> determined the current behaviour is as expected.
>
> To recap: when given a real file, normalizePath() always* returns the full
> absolute path. When given a non-existent file, normalizePath() returns a
> full path on Windows but it returns the input on other systems*. I'd
argue
> that there are benefits to being able to reliably and consistently get a
> full path, regardless of whether the file exists or not. In order to not
> break existing behaviour, I propose adding an argument `absolute = FALSE`
> that will attempt to return an absolute path when the argument is set to
> TRUE. I don't have any evidence for this claim, but I believe that
others
> who use this function would expect, like I did, that an absolute path is
> returned regardless of the file state. I understand the documentation is
> correct because it warns the absolute path may not be returned, but I
> believe it would be a useful feature to support.
>
>
> * I've tested this on Win7, Win10, two versions of MacOS, ubuntu. This
> behaviour may not be true in other OSes
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

Thanks Tomas - I can easily believe that other systems have much less
documentation on this issue! Thanks for you work on this.

I try to stick with base packages when possible, but if the `fs` package
provides this functionality then I'm happy to use that in this case, thanks
Jim.

On Wed, 15 Apr 2020 at 10:06, Jim Hester <james.f.hester at gmail.com>
wrote:
> The fs[1] function `fs::path_abs()` does what I believe you were
> expecting `normalizePath()` to do in this case. e.g.
>
>     setwd("~")
>     normalizePath("foo/bar")
>     #> Warning in normalizePath("foo/bar") :
>     #> path[1]="foo/bar": No such file or directory
>     #> [1] "foo/bar"
>
>     fs::path_abs("foo/bar")
>     #> /Users/jhester/foo/bar
>
> [1]: https://CRAN.R-project.org/package=fs
>
>
> On Tue, Apr 14, 2020 at 1:03 PM Dean Attali <daattali at gmail.com>
wrote:
> >
> > This request stems off a bug report I posted
> > https://bugs.r-project.org/bugzilla/show_bug.cgi?id=17757 where it was
> > determined the current behaviour is as expected.
> >
> > To recap: when given a real file, normalizePath() always* returns the
> full
> > absolute path. When given a non-existent file, normalizePath() returns
a
> > full path on Windows but it returns the input on other systems*.
I'd
> argue
> > that there are benefits to being able to reliably and consistently get
a
> > full path, regardless of whether the file exists or not. In order to
not
> > break existing behaviour, I propose adding an argument `absolute =
FALSE`
> > that will attempt to return an absolute path when the argument is set
to
> > TRUE. I don't have any evidence for this claim, but I believe that
others
> > who use this function would expect, like I did, that an absolute path
is
> > returned regardless of the file state. I understand the documentation
is
> > correct because it warns the absolute path may not be returned, but I
> > believe it would be a useful feature to support.
> >
> >
> > * I've tested this on Win7, Win10, two versions of MacOS, ubuntu.
This
> > behaviour may not be true in other OSes
> >
> >         [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-devel at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
>
	[[alternative HTML version deleted]]