Duncan Murdoch
2021-Jan-27 20:26 UTC
[R] Error when calling (R 4.0.x on Windows) from Python
On 27/01/2021 3:17 p.m., Duncan Murdoch wrote:> On 27/01/2021 3:38 a.m., Martin Maechler wrote: >>>>>>> Martin Maechler >>>>>>> on Tue, 26 Jan 2021 12:37:58 +0100 writes: >> >>>>>>> Marcel Baumgartner >>>>>>> on Tue, 26 Jan 2021 08:55:48 +0100 writes: >> >> >> Dear all, my colleague posted our issue on stackoverflow: >> >> >> Calling R script from Python does not save log file in >> >> version 4 - Stack Overflow >> >> [https://stackoverflow.com/questions/65887485/calling-r-script-from-python-does-not-save-log-file-in-version-4] >> >> >> It is about this kind of call to R: >> >> >> R.exe -f code.R --args "~/file.txt" 1> "~/log.txt" 2>&1". >> >> >> The issue is that the log.txt file is not created when >> >> running R 4.x.x. The same code works perfectly fine with >> >> R 3.6.x. >> >> >> Any idea what's going wrong as of version 4? Regards >> >> Marcel >> >> > Dear Marcel, I think the solution is embarrassingly >> > simple: >> >> >> From the SO post, where she showed a bit more detail than you >> > show here, it's clear you have confused 'R.exe' and >> > 'Rscript.exe' and what you say above is not true: >> >> > 'R.exe' was used for R 3.6.0 but for R 4.0.3, you/she used >> > 'Rscript.exe' instead. >> >> >> > ... as you've noticed now, they do behave differently, >> > indeed! >> >> Well, this was not the solution to their -- Windows-only -- problem. >> The problem *is* indeed visible if they only use R.exe (also >> for R 4.0.3). >> >> I've commented more on the SO issue (see above), >> notably asking for a *minimal* repr.ex. (reproducible example), >> and one *not* using "<YOUR PATH>" and setwd() .. >> > > Isn't this purely a Python or user problem? R shouldn't process > redirection directives like > > 1> "~/log.txt" 2>&1 > > because it's the shell's job to process those. If Python is acting as > the shell, it needs to handle those things. If R was handling the > command viaOops, sent before finishing: If R was handling the command via system() or system2(), it would handle redirection itself. If it was using the Windows-only shell(), it would call cmd.exe (by default) to handle redirection. (This is a difference between R on Windows and R in Unix: in Unix a shell is always used.) Duncan Murdoch
I believe the problem is from svn 77925 in gnuwin/front-ends/rcmdfn.c, which was committed a few days after 3.6.3 was released. Rterm used to put double quotes around a command line argument only if it contained a space, now it double quotes all arguments. It sees shell constructs like "1>" and the following file name as arguments and double quoting them hides them from the shell, leading to this problem. I think we may have to rely on the user supplying quotes as needed instead of blindly adding them. -Bill On Wed, Jan 27, 2021 at 12:28 PM Duncan Murdoch <murdoch.duncan at gmail.com> wrote:> > On 27/01/2021 3:17 p.m., Duncan Murdoch wrote: > > On 27/01/2021 3:38 a.m., Martin Maechler wrote: > >>>>>>> Martin Maechler > >>>>>>> on Tue, 26 Jan 2021 12:37:58 +0100 writes: > >> > >>>>>>> Marcel Baumgartner > >>>>>>> on Tue, 26 Jan 2021 08:55:48 +0100 writes: > >> > >> >> Dear all, my colleague posted our issue on stackoverflow: > >> > >> >> Calling R script from Python does not save log file in > >> >> version 4 - Stack Overflow > >> >> [https://stackoverflow.com/questions/65887485/calling-r-script-from-python-does-not-save-log-file-in-version-4] > >> > >> >> It is about this kind of call to R: > >> > >> >> R.exe -f code.R --args "~/file.txt" 1> "~/log.txt" 2>&1". > >> > >> >> The issue is that the log.txt file is not created when > >> >> running R 4.x.x. The same code works perfectly fine with > >> >> R 3.6.x. > >> > >> >> Any idea what's going wrong as of version 4? Regards > >> >> Marcel > >> > >> > Dear Marcel, I think the solution is embarrassingly > >> > simple: > >> > >> >> From the SO post, where she showed a bit more detail than you > >> > show here, it's clear you have confused 'R.exe' and > >> > 'Rscript.exe' and what you say above is not true: > >> > >> > 'R.exe' was used for R 3.6.0 but for R 4.0.3, you/she used > >> > 'Rscript.exe' instead. > >> > >> > >> > ... as you've noticed now, they do behave differently, > >> > indeed! > >> > >> Well, this was not the solution to their -- Windows-only -- problem. > >> The problem *is* indeed visible if they only use R.exe (also > >> for R 4.0.3). > >> > >> I've commented more on the SO issue (see above), > >> notably asking for a *minimal* repr.ex. (reproducible example), > >> and one *not* using "<YOUR PATH>" and setwd() .. > >> > > > > Isn't this purely a Python or user problem? R shouldn't process > > redirection directives like > > > > 1> "~/log.txt" 2>&1 > > > > because it's the shell's job to process those. If Python is acting as > > the shell, it needs to handle those things. If R was handling the > > command via > > Oops, sent before finishing: > > If R was handling the command via system() or system2(), it would handle > redirection itself. If it was using the Windows-only shell(), it would > call cmd.exe (by default) to handle redirection. (This is a difference > between R on Windows and R in Unix: in Unix a shell is always used.) > > Duncan Murdoch > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.