Keith Jewell
2011-Aug-11 16:29 UTC
[R] UNC windows path beginning with backslashes: normalizePath bug??
Hi,
Back in June I posted the message below, but had no replies. I've made a
little progress since then so this is to update anyone interested (!) and to
ask for comments.
Brief problem statement:
Under Windows, some parts of R don't handle UNC paths beginning with
backslashes. Specifically
a) Sys.glob() fails to find some files breaking (e.g.) Rcmdr plugins
Sys.glob(file.path(.libPaths(), "*/etc/menus.txt"))
fails to find files which are there
b) update.packages(ask='graphics') fails when copying the updates into
the
destination folders
In Renviron.site I define the site library with forward slashes, not
backslashes thus...
R_LIBS_SITE=//campden/shares/workgroup/stats/R/library/%v
... but the startup process seems to replace them with forward slashes.
I guess because .libPaths with a 'new' argument calls normalizePath
which
changes leading slashes to backslashes, even with
winslash="/"> normalizePath("//campden/shares/workgroup/stats/R/library",
winslash="/")
[1] "\\\\campden/shares/workgroup/Stats/R/library"
I've corrected (??) this by inserting a line into Rprofile.site
assign(".lib.loc", gsub("\\", "/", .libPaths(),
fixed=TRUE),
env=environment(.libPaths))
That seems to fix problem (a) above, which was affecting a number of users.
But have I broken anything else?
I'm still experiencing problem (b).
I'm the only person on site who updates packages so I've mapped a drive
letter (L:) and in my own .Rprofile I have a line
assign(".lib.loc",
sub("//campden/shares/workgroup/Stats", "L:",
.libPaths(), ignore.case = TRUE), env=environment(.libPaths))
So that's OK as far as it goes, but it's all a bit messy!
If .libPaths is called with a 'new' argument it will breaks things
again.
normalizePath seems to produce paths that don't work with Sys.glob.
I have the feeling I'm being silly and making hard work of all this.
Any comments? Suggestions?
Best regards, and thanks in advance/
Keith Jewell
"Keith Jewell" <k.jewell at campden.co.uk> wrote in message
news:...> Hi,
>
> Back in 2010 I had a problem with 'update.packages()', which I
worked
> around by mapping a drive letter to a UNC path [described in
> <http://finzi.psych.upenn.edu/Rhelp10/2010-February/229820.html> but
my
> current workaround is
> assign(".lib.loc", sub("\\\\\\\\Server02/stats",
"L:", .libPaths(),
> ignore.case = TRUE), env=environment(.libPaths))].
>
> More recently a colleague had problems using the 'FactoMineR' plug
in for
> the Rcmdr package;
> a) directly loading 'RcmdrPlugin.FactoMineR' opened and
"crashed" R
> Commander;
> b) opening R Commander without FactoMiner, the Tools option 'Load Rcmdr
> plug-in(s)...' was greyed out.
>
> It transpired that in .libPaths() the path to the library holding
> 'RcmdrPlugin.FactoMineR' was specified as a UNC address:
> \\\\Server02/stats/R/library/2.13>. Mapping a virtual drive letter (e.g.
> L:) and specifying the path in .libPaths() as a 'local file system'
(LFS)
> address <L:/R/library/2.13> fixed the problem.
>
> I contacted Professor Fox (maintainer of Rcmdr) who told me that Rcmdr
> finds plug-in packages via the command
> plugins <- unlist(lapply(.libPaths(), function(x) Sys.glob(file.path(x,
> "*/etc/menus.txt"))))
> Because file.path and Sys.glob are both vectorised I think (but am not
> certain) that this could be simplified to:
> plugins <- Sys.glob(file.path(.libPaths(),
"*/etc/menus.txt"))
> but that's by the way, the problem seems to lie in Sys.glob under
Windows
> operating systems.
>
> I note that 'help(Sys.glob)' on my Windows system differs from
> <http://finzi.psych.upenn.edu/R/library/base/html/Sys.glob.html>.
> The latter says "For precise details, see your system's
documentation on
> the glob system call. There is a POSIX 1003.2 standard <snip> The
rest
> of these details are indicative (and based on the POSIX standard)".
> On Windows "The glob system call is not part of Windows, and we supply
a
> partial emulation. <snip> An attempt is made to handle UNC paths
starting
> with a double backslash" which doesn't really inspire confidence.
>
> This was discussed in a 2009 R-devel thread starting here
> <https://stat.ethz.ch/pipermail/r-devel/2009-June/053879.html>, but
the
> patch proposed in that thread seems not to have been implemented (??).
>
> Trying to avoid Sys.glob in the Rcmdr application I came up with this:
> list.files(path=file.path(list.files(path=.libPaths(),
> full.names=TRUE), "etc"), pattern="^menus\\.txt$",
full.names=TRUE)
> It seems to give identical results to Sys.glob for mapped drives, works
> with UNC paths in Windows, and seems quite fast.
>
> So my questions relate to diagnosis, prognosis, and prescription (cure?).
>
> 1) Diagnosis: Am I correct that my problem(s) originate in the
"partial
> emulation" of glob in Windows.
>
> 2) Prognosis: If so, is there any likelihood that the emulation will
> improve in the near future?
>
> 3) Prescription: If not:
>
> a) is assign(".lib.loc", sub("\\\\\\\\Server02/stats",
"L:", .libPaths(),
> ignore.case = TRUE), env=environment(.libPaths))
> a reasonable workaround in a specific case?
>
> b) is list.files(path=file.path(list.files(path=.libPaths(),
> full.names=TRUE), "etc"), pattern="^menus\\.txt$",
full.names=TRUE)
> a reasonable replacement for the Sys.glob() construction in Rcmdr? I
don't
> want to suggest to Prof. Fox an amendment which fixes my problem but
> 'breaks' it for others!
>
> Thanks in advance,
>
> Keith Jewell
>
> R version 2.13.0 (2011-04-13)
> Platform: i386-pc-mingw32/i386 (32-bit)
>
> locale:
> [1] LC_COLLATE=English_United Kingdom.1252
> [2] LC_CTYPE=English_United Kingdom.1252
> [3] LC_MONETARY=English_United Kingdom.1252
> [4] LC_NUMERIC=C
> [5] LC_TIME=English_United Kingdom.1252
>
> attached base packages:
> [1] datasets grDevices splines graphics stats utils tcltk
> [8] tools methods base
>
>
>
Keith Jewell
2011-Aug-11 16:29 UTC
[R] UNC windows path beginning with backslashes: normalizePath bug??
Hi,
Back in June I posted the message below, but had no replies. I've made a
little progress since then so this is to update anyone interested (!) and to
ask for comments.
Brief problem statement:
Under Windows, some parts of R don't handle UNC paths beginning with
backslashes. Specifically
a) Sys.glob() fails to find some files breaking (e.g.) Rcmdr plugins
Sys.glob(file.path(.libPaths(), "*/etc/menus.txt"))
fails to find files which are there
b) update.packages(ask='graphics') fails when copying the updates into
the
destination folders
In Renviron.site I define the site library with forward slashes, not
backslashes thus...
R_LIBS_SITE=//campden/shares/workgroup/stats/R/library/%v
... but the startup process seems to replace them with forward slashes.
I guess because .libPaths with a 'new' argument calls normalizePath
which
changes leading slashes to backslashes, even with
winslash="/"> normalizePath("//campden/shares/workgroup/stats/R/library",
winslash="/")
[1] "\\\\campden/shares/workgroup/Stats/R/library"
I've corrected (??) this by inserting a line into Rprofile.site
assign(".lib.loc", gsub("\\", "/", .libPaths(),
fixed=TRUE),
env=environment(.libPaths))
That seems to fix problem (a) above, which was affecting a number of users.
But have I broken anything else?
I'm still experiencing problem (b).
I'm the only person on site who updates packages so I've mapped a drive
letter (L:) and in my own .Rprofile I have a line
assign(".lib.loc",
sub("//campden/shares/workgroup/Stats", "L:",
.libPaths(), ignore.case = TRUE), env=environment(.libPaths))
So that's OK as far as it goes, but it's all a bit messy!
If .libPaths is called with a 'new' argument it will breaks things
again.
normalizePath seems to produce paths that don't work with Sys.glob.
I have the feeling I'm being silly and making hard work of all this.
Any comments? Suggestions?
Best regards, and thanks in advance/
Keith Jewell
"Keith Jewell" <k.jewell at campden.co.uk> wrote in message
news:...> Hi,
>
> Back in 2010 I had a problem with 'update.packages()', which I
worked
> around by mapping a drive letter to a UNC path [described in
> <http://finzi.psych.upenn.edu/Rhelp10/2010-February/229820.html> but
my
> current workaround is
> assign(".lib.loc", sub("\\\\\\\\Server02/stats",
"L:", .libPaths(),
> ignore.case = TRUE), env=environment(.libPaths))].
>
> More recently a colleague had problems using the 'FactoMineR' plug
in for
> the Rcmdr package;
> a) directly loading 'RcmdrPlugin.FactoMineR' opened and
"crashed" R
> Commander;
> b) opening R Commander without FactoMiner, the Tools option 'Load Rcmdr
> plug-in(s)...' was greyed out.
>
> It transpired that in .libPaths() the path to the library holding
> 'RcmdrPlugin.FactoMineR' was specified as a UNC address:
> \\\\Server02/stats/R/library/2.13>. Mapping a virtual drive letter (e.g.
> L:) and specifying the path in .libPaths() as a 'local file system'
(LFS)
> address <L:/R/library/2.13> fixed the problem.
>
> I contacted Professor Fox (maintainer of Rcmdr) who told me that Rcmdr
> finds plug-in packages via the command
> plugins <- unlist(lapply(.libPaths(), function(x) Sys.glob(file.path(x,
> "*/etc/menus.txt"))))
> Because file.path and Sys.glob are both vectorised I think (but am not
> certain) that this could be simplified to:
> plugins <- Sys.glob(file.path(.libPaths(),
"*/etc/menus.txt"))
> but that's by the way, the problem seems to lie in Sys.glob under
Windows
> operating systems.
>
> I note that 'help(Sys.glob)' on my Windows system differs from
> <http://finzi.psych.upenn.edu/R/library/base/html/Sys.glob.html>.
> The latter says "For precise details, see your system's
documentation on
> the glob system call. There is a POSIX 1003.2 standard <snip> The
rest
> of these details are indicative (and based on the POSIX standard)".
> On Windows "The glob system call is not part of Windows, and we supply
a
> partial emulation. <snip> An attempt is made to handle UNC paths
starting
> with a double backslash" which doesn't really inspire confidence.
>
> This was discussed in a 2009 R-devel thread starting here
> <https://stat.ethz.ch/pipermail/r-devel/2009-June/053879.html>, but
the
> patch proposed in that thread seems not to have been implemented (??).
>
> Trying to avoid Sys.glob in the Rcmdr application I came up with this:
> list.files(path=file.path(list.files(path=.libPaths(),
> full.names=TRUE), "etc"), pattern="^menus\\.txt$",
full.names=TRUE)
> It seems to give identical results to Sys.glob for mapped drives, works
> with UNC paths in Windows, and seems quite fast.
>
> So my questions relate to diagnosis, prognosis, and prescription (cure?).
>
> 1) Diagnosis: Am I correct that my problem(s) originate in the
"partial
> emulation" of glob in Windows.
>
> 2) Prognosis: If so, is there any likelihood that the emulation will
> improve in the near future?
>
> 3) Prescription: If not:
>
> a) is assign(".lib.loc", sub("\\\\\\\\Server02/stats",
"L:", .libPaths(),
> ignore.case = TRUE), env=environment(.libPaths))
> a reasonable workaround in a specific case?
>
> b) is list.files(path=file.path(list.files(path=.libPaths(),
> full.names=TRUE), "etc"), pattern="^menus\\.txt$",
full.names=TRUE)
> a reasonable replacement for the Sys.glob() construction in Rcmdr? I
don't
> want to suggest to Prof. Fox an amendment which fixes my problem but
> 'breaks' it for others!
>
> Thanks in advance,
>
> Keith Jewell
>
> R version 2.13.0 (2011-04-13)
> Platform: i386-pc-mingw32/i386 (32-bit)
>
> locale:
> [1] LC_COLLATE=English_United Kingdom.1252
> [2] LC_CTYPE=English_United Kingdom.1252
> [3] LC_MONETARY=English_United Kingdom.1252
> [4] LC_NUMERIC=C
> [5] LC_TIME=English_United Kingdom.1252
>
> attached base packages:
> [1] datasets grDevices splines graphics stats utils tcltk
> [8] tools methods base
>
>
>
Uwe Ligges
2011-Aug-11 16:53 UTC
[R] UNC windows path beginning with backslashes: normalizePath bug??
This is extremely tricky since Windows does not always accept "//" rather than "\\". Additionally, there is not implemented system call in Windows, hence ?Sys.glob tells us a "partial emulation" is provided and "An attempt is made to handle UNC paths starting with a double backslash." As you have seenm this does not work everywhere, therefore it is advisable to run R from mapped drives - as I am doing in the network of our university for 13 years without problems now. Best, Uwe Ligges On 11.08.2011 18:29, Keith Jewell wrote:> Hi, > > Back in June I posted the message below, but had no replies. I've made a > little progress since then so this is to update anyone interested (!) and to > ask for comments. > > Brief problem statement: > Under Windows, some parts of R don't handle UNC paths beginning with > backslashes. Specifically > a) Sys.glob() fails to find some files breaking (e.g.) Rcmdr plugins > Sys.glob(file.path(.libPaths(), "*/etc/menus.txt")) > fails to find files which are there > > b) update.packages(ask='graphics') fails when copying the updates into the > destination folders > > In Renviron.site I define the site library with forward slashes, not > backslashes thus... > R_LIBS_SITE=//campden/shares/workgroup/stats/R/library/%v > ... but the startup process seems to replace them with forward slashes. > I guess because .libPaths with a 'new' argument calls normalizePath which > changes leading slashes to backslashes, even with winslash="/" >> normalizePath("//campden/shares/workgroup/stats/R/library", winslash="/") > [1] "\\\\campden/shares/workgroup/Stats/R/library" > > I've corrected (??) this by inserting a line into Rprofile.site > assign(".lib.loc", gsub("\\", "/", .libPaths(), fixed=TRUE), > env=environment(.libPaths)) > That seems to fix problem (a) above, which was affecting a number of users. > But have I broken anything else? > > I'm still experiencing problem (b). > I'm the only person on site who updates packages so I've mapped a drive > letter (L:) and in my own .Rprofile I have a line > assign(".lib.loc", sub("//campden/shares/workgroup/Stats", "L:", > .libPaths(), ignore.case = TRUE), env=environment(.libPaths)) > > So that's OK as far as it goes, but it's all a bit messy! > If .libPaths is called with a 'new' argument it will breaks things again. > normalizePath seems to produce paths that don't work with Sys.glob. > > I have the feeling I'm being silly and making hard work of all this. > > Any comments? Suggestions? > > Best regards, and thanks in advance/ > > Keith Jewell > > "Keith Jewell"<k.jewell at campden.co.uk> wrote in message news:... >> Hi, >> >> Back in 2010 I had a problem with 'update.packages()', which I worked >> around by mapping a drive letter to a UNC path [described in >> <http://finzi.psych.upenn.edu/Rhelp10/2010-February/229820.html> but my >> current workaround is >> assign(".lib.loc", sub("\\\\\\\\Server02/stats", "L:", .libPaths(), >> ignore.case = TRUE), env=environment(.libPaths))]. >> >> More recently a colleague had problems using the 'FactoMineR' plug in for >> the Rcmdr package; >> a) directly loading 'RcmdrPlugin.FactoMineR' opened and "crashed" R >> Commander; >> b) opening R Commander without FactoMiner, the Tools option 'Load Rcmdr >> plug-in(s)...' was greyed out. >> >> It transpired that in .libPaths() the path to the library holding >> 'RcmdrPlugin.FactoMineR' was specified as a UNC address: >> \\\\Server02/stats/R/library/2.13>. Mapping a virtual drive letter (e.g. >> L:) and specifying the path in .libPaths() as a 'local file system' (LFS) >> address<L:/R/library/2.13> fixed the problem. >> >> I contacted Professor Fox (maintainer of Rcmdr) who told me that Rcmdr >> finds plug-in packages via the command >> plugins<- unlist(lapply(.libPaths(), function(x) Sys.glob(file.path(x, >> "*/etc/menus.txt")))) >> Because file.path and Sys.glob are both vectorised I think (but am not >> certain) that this could be simplified to: >> plugins<- Sys.glob(file.path(.libPaths(), "*/etc/menus.txt")) >> but that's by the way, the problem seems to lie in Sys.glob under Windows >> operating systems. >> >> I note that 'help(Sys.glob)' on my Windows system differs from >> <http://finzi.psych.upenn.edu/R/library/base/html/Sys.glob.html>. >> The latter says "For precise details, see your system's documentation on >> the glob system call. There is a POSIX 1003.2 standard<snip> The rest >> of these details are indicative (and based on the POSIX standard)". >> On Windows "The glob system call is not part of Windows, and we supply a >> partial emulation.<snip> An attempt is made to handle UNC paths starting >> with a double backslash" which doesn't really inspire confidence. >> >> This was discussed in a 2009 R-devel thread starting here >> <https://stat.ethz.ch/pipermail/r-devel/2009-June/053879.html>, but the >> patch proposed in that thread seems not to have been implemented (??). >> >> Trying to avoid Sys.glob in the Rcmdr application I came up with this: >> list.files(path=file.path(list.files(path=.libPaths(), >> full.names=TRUE), "etc"), pattern="^menus\\.txt$", full.names=TRUE) >> It seems to give identical results to Sys.glob for mapped drives, works >> with UNC paths in Windows, and seems quite fast. >> >> So my questions relate to diagnosis, prognosis, and prescription (cure?). >> >> 1) Diagnosis: Am I correct that my problem(s) originate in the "partial >> emulation" of glob in Windows. >> >> 2) Prognosis: If so, is there any likelihood that the emulation will >> improve in the near future? >> >> 3) Prescription: If not: >> >> a) is assign(".lib.loc", sub("\\\\\\\\Server02/stats", "L:", .libPaths(), >> ignore.case = TRUE), env=environment(.libPaths)) >> a reasonable workaround in a specific case? >> >> b) is list.files(path=file.path(list.files(path=.libPaths(), >> full.names=TRUE), "etc"), pattern="^menus\\.txt$", full.names=TRUE) >> a reasonable replacement for the Sys.glob() construction in Rcmdr? I don't >> want to suggest to Prof. Fox an amendment which fixes my problem but >> 'breaks' it for others! >> >> Thanks in advance, >> >> Keith Jewell >> >> R version 2.13.0 (2011-04-13) >> Platform: i386-pc-mingw32/i386 (32-bit) >> >> locale: >> [1] LC_COLLATE=English_United Kingdom.1252 >> [2] LC_CTYPE=English_United Kingdom.1252 >> [3] LC_MONETARY=English_United Kingdom.1252 >> [4] LC_NUMERIC=C >> [5] LC_TIME=English_United Kingdom.1252 >> >> attached base packages: >> [1] datasets grDevices splines graphics stats utils tcltk >> [8] tools methods base >> >> >> > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.