R devel - May 2015 - How best to get around shadowing of executables by system()'s prepending of directories to Windows' PATH?

My question:

On Windows, R's system() command prepends several directories to those
in the Windows Path variable.
>From ?system
     The search path for 'command' may be system-dependent: it will
     include the R 'bin' directory, the working directory and the
     Windows system directories before 'PATH'.

This shadows any executables on the Path that share a name with, for
example, one of the Windows commands.

What should I do when I'd really like (the equivalent of) a call
passed to system() that would be executed using the same Path that
you'd get if working directly at the Windows command line? Is there a
recommended workaround for situtations like this? (It _seems_ like it
would be handy if system() et al. included an additional argument that
optionally disabled the prepending of those extra directories, to give
Windows users full control of the path seen by system(). Would adding
such an argument have undesirable ramifications?)


Motivation and reproducible example:

I'm motivated here by a desire to use the function plotdiff() from
Paul Murrell's gridGraphics package on my Windows laptop.  Getting
that to work will require a few code fixes, of which the masking of
ImageMagick's convert.exe by that in the C:/Windows/System32 seems to
be the most challenging. plotdiff() relies on system2() calls to
ImageMagick's 'convert'  function, as well as a call to
Sys.which(c("convert", "compare")) that tests for the
presence of
ImageMagick on the Path. Even  if ImageMagick is placed early on the
Path, though, both calls to Sys.which() and system2() find Windows'
convert command  (which "Converts FAT volumes to NTFS") rather than
ImageMagick's convert.


Here's a reproducible example that shows what I'm seeing:

    ## In R, make a pdf
    pdf("a.pdf")
    plot(rnorm(99), col="red")
    dev.off()

    ## At Windows cmd command line
    where convert
    ## C:\Program Files\ImageMagick-6.8.8-Q16\convert.exe
    ## C:\Windows\System32\convert.exe
    convert -density 100x100 a.pdf a.png

    ## From R

    ## Unqualified references to convert find the 'wrong' one
    Sys.which("convert")
    ##                               convert
    ## "C:\\Windows\\system32\\convert.exe"
     system2("convert",  "-density 100x100 a.pdf b.png")
    ## Invalid Parameter - 100x100
    ## Warning message:
    ## running command '"convert" -density 100x100 a.pdf
b.png' had status 4

    ## A fully qualified reference does work
    system2("C:/Program Files/ImageMagick-6.8.8-Q16/convert",
"-density 100x100 a.pdf b.png")

+1 I have exactly the same problem.

Regards,
Yihui
--
Yihui Xie <xieyihui at gmail.com>
Web: http://yihui.name


On Mon, May 18, 2015 at 12:29 PM, Josh O'Brien <joshmobrien at
gmail.com> wrote:
> My question:
>
> On Windows, R's system() command prepends several directories to those
> in the Windows Path variable.
>
> >From ?system
>
>      The search path for 'command' may be system-dependent: it will
>      include the R 'bin' directory, the working directory and the
>      Windows system directories before 'PATH'.
>
> This shadows any executables on the Path that share a name with, for
> example, one of the Windows commands.
>
> What should I do when I'd really like (the equivalent of) a call
> passed to system() that would be executed using the same Path that
> you'd get if working directly at the Windows command line? Is there a
> recommended workaround for situtations like this? (It _seems_ like it
> would be handy if system() et al. included an additional argument that
> optionally disabled the prepending of those extra directories, to give
> Windows users full control of the path seen by system(). Would adding
> such an argument have undesirable ramifications?)
>
>
> Motivation and reproducible example:
>
> I'm motivated here by a desire to use the function plotdiff() from
> Paul Murrell's gridGraphics package on my Windows laptop.  Getting
> that to work will require a few code fixes, of which the masking of
> ImageMagick's convert.exe by that in the C:/Windows/System32 seems to
> be the most challenging. plotdiff() relies on system2() calls to
> ImageMagick's 'convert'  function, as well as a call to
> Sys.which(c("convert", "compare")) that tests for the
presence of
> ImageMagick on the Path. Even  if ImageMagick is placed early on the
> Path, though, both calls to Sys.which() and system2() find Windows'
> convert command  (which "Converts FAT volumes to NTFS") rather
than
> ImageMagick's convert.
>
>
> Here's a reproducible example that shows what I'm seeing:
>
>     ## In R, make a pdf
>     pdf("a.pdf")
>     plot(rnorm(99), col="red")
>     dev.off()
>
>     ## At Windows cmd command line
>     where convert
>     ## C:\Program Files\ImageMagick-6.8.8-Q16\convert.exe
>     ## C:\Windows\System32\convert.exe
>     convert -density 100x100 a.pdf a.png
>
>     ## From R
>
>     ## Unqualified references to convert find the 'wrong' one
>     Sys.which("convert")
>     ##                               convert
>     ## "C:\\Windows\\system32\\convert.exe"
>      system2("convert",  "-density 100x100 a.pdf
b.png")
>     ## Invalid Parameter - 100x100
>     ## Warning message:
>     ## running command '"convert" -density 100x100 a.pdf
b.png' had status 4
>
>     ## A fully qualified reference does work
>     system2("C:/Program Files/ImageMagick-6.8.8-Q16/convert",
> "-density 100x100 a.pdf b.png")
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

You probably already know, but you can at least work around it as:

Sys.which2 <- function(cmd) {
  stopifnot(length(cmd) == 1)
  if (.Platform$OS.type == "windows") {
    suppressWarnings({
      pathname <- shell(sprintf("where %s 2> NUL", cmd),
intern=TRUE)[1]
   })
   if (!is.na(pathname)) return(setNames(pathname, cmd))
  }
  Sys.which(cmd)
}

(it falls back to Sys.which() if 'where %s' doesn't give anything)

> Sys.which2("convert")
                                                convert
"C:\\Program Files\\ImageMagick-6.8.3-Q16\\convert.exe"
> Sys.which("convert")
                             convert
"C:\\Windows\\system32\\convert.exe"

/Henrik

On Mon, May 18, 2015 at 11:08 AM, Yihui Xie <xie at yihui.name>
wrote:
> +1 I have exactly the same problem.
>
> Regards,
> Yihui
> --
> Yihui Xie <xieyihui at gmail.com>
> Web: http://yihui.name
>
>
> On Mon, May 18, 2015 at 12:29 PM, Josh O'Brien <joshmobrien at
gmail.com> wrote:
>> My question:
>>
>> On Windows, R's system() command prepends several directories to
those
>> in the Windows Path variable.
>>
>> >From ?system
>>
>>      The search path for 'command' may be system-dependent: it
will
>>      include the R 'bin' directory, the working directory and
the
>>      Windows system directories before 'PATH'.
>>
>> This shadows any executables on the Path that share a name with, for
>> example, one of the Windows commands.
>>
>> What should I do when I'd really like (the equivalent of) a call
>> passed to system() that would be executed using the same Path that
>> you'd get if working directly at the Windows command line? Is there
a
>> recommended workaround for situtations like this? (It _seems_ like it
>> would be handy if system() et al. included an additional argument that
>> optionally disabled the prepending of those extra directories, to give
>> Windows users full control of the path seen by system(). Would adding
>> such an argument have undesirable ramifications?)
>>
>>
>> Motivation and reproducible example:
>>
>> I'm motivated here by a desire to use the function plotdiff() from
>> Paul Murrell's gridGraphics package on my Windows laptop.  Getting
>> that to work will require a few code fixes, of which the masking of
>> ImageMagick's convert.exe by that in the C:/Windows/System32 seems
to
>> be the most challenging. plotdiff() relies on system2() calls to
>> ImageMagick's 'convert'  function, as well as a call to
>> Sys.which(c("convert", "compare")) that tests for
the presence of
>> ImageMagick on the Path. Even  if ImageMagick is placed early on the
>> Path, though, both calls to Sys.which() and system2() find Windows'
>> convert command  (which "Converts FAT volumes to NTFS")
rather than
>> ImageMagick's convert.
>>
>>
>> Here's a reproducible example that shows what I'm seeing:
>>
>>     ## In R, make a pdf
>>     pdf("a.pdf")
>>     plot(rnorm(99), col="red")
>>     dev.off()
>>
>>     ## At Windows cmd command line
>>     where convert
>>     ## C:\Program Files\ImageMagick-6.8.8-Q16\convert.exe
>>     ## C:\Windows\System32\convert.exe
>>     convert -density 100x100 a.pdf a.png
>>
>>     ## From R
>>
>>     ## Unqualified references to convert find the 'wrong' one
>>     Sys.which("convert")
>>     ##                               convert
>>     ## "C:\\Windows\\system32\\convert.exe"
>>      system2("convert",  "-density 100x100 a.pdf
b.png")
>>     ## Invalid Parameter - 100x100
>>     ## Warning message:
>>     ## running command '"convert" -density 100x100 a.pdf
b.png' had status 4
>>
>>     ## A fully qualified reference does work
>>     system2("C:/Program Files/ImageMagick-6.8.8-Q16/convert",
>> "-density 100x100 a.pdf b.png")
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

On Mon, May 18, 2015 at 10:29 AM, Josh O'Brien <joshmobrien at
gmail.com> wrote:
> My question:
>
> On Windows, R's system() command prepends several directories to those
> in the Windows Path variable.
>
> From ?system
>
>      The search path for 'command' may be system-dependent: it will
>      include the R 'bin' directory, the working directory and the
>      Windows system directories before 'PATH'.
>
> This shadows any executables on the Path that share a name with, for
> example, one of the Windows commands.
>
> What should I do when I'd really like (the equivalent of) a call
> passed to system() that would be executed using the same Path that
> you'd get if working directly at the Windows command line? Is there a
It appears that "use shell() instead of system()" is a reasonable, if
narrow, answer to this question.

Doing:

    shell("convert -density 100x100 a.pdf b.png")

constructs (and then uses system() to call) the following command:

    "C:\\Windows\\system32\\cmd.exe /c convert -density 100x100 a.pdf
b.png"

which gives you just what you'd get if typing directly at the command line.
> recommended workaround for situtations like this? (It _seems_ like it
> would be handy if system() et al. included an additional argument that
> optionally disabled the prepending of those extra directories, to give
> Windows users full control of the path seen by system(). Would adding
> such an argument have undesirable ramifications?)
>
>
> Motivation and reproducible example:
>
> I'm motivated here by a desire to use the function plotdiff() from
> Paul Murrell's gridGraphics package on my Windows laptop.  Getting
> that to work will require a few code fixes, of which the masking of
> ImageMagick's convert.exe by that in the C:/Windows/System32 seems to
> be the most challenging. plotdiff() relies on system2() calls to
> ImageMagick's 'convert'  function, as well as a call to
> Sys.which(c("convert", "compare")) that tests for the
presence of
> ImageMagick on the Path. Even  if ImageMagick is placed early on the
> Path, though, both calls to Sys.which() and system2() find Windows'
> convert command  (which "Converts FAT volumes to NTFS") rather
than
> ImageMagick's convert.
>
>
> Here's a reproducible example that shows what I'm seeing:
>
>     ## In R, make a pdf
>     pdf("a.pdf")
>     plot(rnorm(99), col="red")
>     dev.off()
>
>     ## At Windows cmd command line
>     where convert
>     ## C:\Program Files\ImageMagick-6.8.8-Q16\convert.exe
>     ## C:\Windows\System32\convert.exe
>     convert -density 100x100 a.pdf a.png
>
>     ## From R
>
>     ## Unqualified references to convert find the 'wrong' one
>     Sys.which("convert")
>     ##                               convert
>     ## "C:\\Windows\\system32\\convert.exe"
>      system2("convert",  "-density 100x100 a.pdf
b.png")
>     ## Invalid Parameter - 100x100
>     ## Warning message:
>     ## running command '"convert" -density 100x100 a.pdf
b.png' had status 4
>
>     ## A fully qualified reference does work
>     system2("C:/Program Files/ImageMagick-6.8.8-Q16/convert",
> "-density 100x100 a.pdf b.png")

This is the code I use in my 'SuperCurve' R-Forge package:


##-------------------------------------------------------------------------
----
## Merge output graphs with source tiff file, save it as JPG file
.mergeGraphsAndImage <- function(antibody,
                                 prefix,
                                 outputdir,
                                 tiff) {
    ## Check arguments
    stopifnot(is.character(antibody)  && length(antibody) == 1)
    stopifnot(is.character(prefix)    && length(prefix) == 1)
    stopifnot(is.character(outputdir) && length(outputdir) == 1)
    stopifnot(is.character(tiff)      && length(tiff) == 1)

    ## Begin processing
    filename <- sprintf("%s_%s_1.png", prefix, antibody)
    pg1 <- file.path(outputdir, .portableFilename(filename))

    filename <- sprintf("%s_%s_2.png", prefix, antibody)
    pg2 <- file.path(outputdir, .portableFilename(filename))

    filename <- sprintf("%s.jpg", antibody)
    output <- file.path(outputdir, .portableFilename(filename))

    ## Use ImageMagick 'convert' binary to perform merge
    command <- paste("convert",
                     shQuote(pg1),
                     shQuote(pg2),
                     "+append",
                     shQuote(tiff),
                     "-append",
                     "-quality 100",
                     shQuote(output))
    rc <- switch(EXPR=.Platform$OS.type,
                 unix=system(command),
                 windows=shell(command),
                 stop(sprintf("unrecognized operating system family
%s",
                              sQuote(.Platform$OS.type))))
    #cat("rc =", rc, ", command:", command, "\n")

    rc
}



Additionally, the package uses the .onLoad() method to verify
WHICH 'convert' would be used and alert user if missing (or
DOS command would be used instead of ImageMagick binary).


##-------------------------------------------------------------------------
----
.onLoad <- function(libname, pkgname) {

    
##-------------------------------------------------------------------------
    ## Preflight check use of ImageMagick 'convert' binary
    preflightCheck <- function() {
        command <- "convert --version"
        tryCatch({
                output <- switch(EXPR=.Platform$OS.type,
                                 unix=system(command,
                                             intern=TRUE,
                                             ignore.stderr=TRUE),
                                 windows=shell(command,
                                               intern=TRUE,
                                               ignore.stderr=TRUE),
                                 "")
                grepl("ImageMagick", output[1], fixed=TRUE)
            },
            error=function(e) {
                FALSE
            })
    }


    if (!preflightCheck()) {
        warning(sprintf("ImageMagick executable %s not installed or
unavailable via PATH",
                        sQuote("convert")),
                call.=FALSE)
    }

    ## [SNIP unrelated additional code]
}




On 5/18/15 12:29 PM, "Josh O'Brien" <joshmobrien at
gmail.com> wrote:
>My question:
>
>On Windows, R's system() command prepends several directories to those
>in the Windows Path variable.
>
>>From ?system
>
>     The search path for 'command' may be system-dependent: it will
>     include the R 'bin' directory, the working directory and the
>     Windows system directories before 'PATH'.
>
>This shadows any executables on the Path that share a name with, for
>example, one of the Windows commands.
>
>What should I do when I'd really like (the equivalent of) a call
>passed to system() that would be executed using the same Path that
>you'd get if working directly at the Windows command line? Is there a
>recommended workaround for situtations like this? (It _seems_ like it
>would be handy if system() et al. included an additional argument that
>optionally disabled the prepending of those extra directories, to give
>Windows users full control of the path seen by system(). Would adding
>such an argument have undesirable ramifications?)
>
>
>Motivation and reproducible example:
>
>I'm motivated here by a desire to use the function plotdiff() from
>Paul Murrell's gridGraphics package on my Windows laptop.  Getting
>that to work will require a few code fixes, of which the masking of
>ImageMagick's convert.exe by that in the C:/Windows/System32 seems to
>be the most challenging. plotdiff() relies on system2() calls to
>ImageMagick's 'convert'  function, as well as a call to
>Sys.which(c("convert", "compare")) that tests for the
presence of
>ImageMagick on the Path. Even  if ImageMagick is placed early on the
>Path, though, both calls to Sys.which() and system2() find Windows'
>convert command  (which "Converts FAT volumes to NTFS") rather
than
>ImageMagick's convert.
>
>
>Here's a reproducible example that shows what I'm seeing:
>
>    ## In R, make a pdf
>    pdf("a.pdf")
>    plot(rnorm(99), col="red")
>    dev.off()
>
>    ## At Windows cmd command line
>    where convert
>    ## C:\Program Files\ImageMagick-6.8.8-Q16\convert.exe
>    ## C:\Windows\System32\convert.exe
>    convert -density 100x100 a.pdf a.png
>
>    ## From R
>
>    ## Unqualified references to convert find the 'wrong' one
>    Sys.which("convert")
>    ##                               convert
>    ## "C:\\Windows\\system32\\convert.exe"
>     system2("convert",  "-density 100x100 a.pdf b.png")
>    ## Invalid Parameter - 100x100
>    ## Warning message:
>    ## running command '"convert" -density 100x100 a.pdf
b.png' had
>status 4
>
>    ## A fully qualified reference does work
>    system2("C:/Program Files/ImageMagick-6.8.8-Q16/convert",
>"-density 100x100 a.pdf b.png")
>
>______________________________________________
>R-devel at r-project.org mailing list
>https://stat.ethz.ch/mailman/listinfo/r-devel