I would have used source("clipboard") on systems which support it
(Tomas
has confirmed it works on Linux). See ?file.
The macOS equivalent source(pipe("pbpaste")) also works.
On 14/06/2021 11:06, Cesko Voeten wrote:> Making it 1024 times larger gives:
>
> installing 'sysdata.rda'
> Error: segfault from C stack overflow
>
> Making it only 4 times larger provides a usable R. In my test case of
> copying&pasting mgcv::gam, I observe the same visual corruption at the
> prompt as before, but when pressing return it has actually been received
> correctly. My real-world problem involved a file 33KiB in size, which -
> as expected, since 16KiB < 33KiB - still has the same problem as before.
>
> I know nothing about readline, but I presume that there is no way for
> this buffer size to be dynamically resized at run time. In that case,
> maybe R should simply force-disable readline's bracketed paste? By the
> way, according to readline's changelog, this does indeed seem to be a
> feature that changed (viz. was enabled in more places) from readline-8.0
> to readline-8.1.
>
> Finally, please disregard my earlier comment about vim and nano working
> just fine. They do, but they don't actually use readline (according to
> ldd), so don't provide a valid comparison.
>
> Thanks for your efforts!
> Cesko
>
> On 14-06-2021 at 08:33, Tomas Kalibera wrote:
>> Thanks, Cesko, for more debugging. As you are already compiling the
>> code, could you please try increasing CONSOLE_BUFFER_SIZE in
>> ./include/Defn.h from 4096 to some very large value (e.g. 1024 times),
>> rebuild R and check if the problems (not all bytes received correctly,
>> visual corruption) go away for texts of the size you looked at before?
>>
>>
>> Thanks,
>> Tomas
>>
>>
>>
>> On 6/13/21 10:59 AM, Voeten, C.C. wrote:
>>>
>>> Thanks for looking into this! I've just compiled today's
R-devel
>>> snapshot, and it shows the same issue. extSoftVersion() from that
build:
>>>
>>> ???????????????????????????????????????????? zlib
>>> ???????????????????????????????????????? "1.2.11"
>>> ??????????????????????????????????????????? bzlib
>>> ???????????????????????????? "1.0.8, 13-Jul-2019"
>>> ?????????????????????????????????????????????? xz
>>> ????????????????????????????????????????? "5.2.5"
>>> ???????????????????????????????????????????? PCRE
>>> ?????????????????????????????? "10.37 2021-05-26"
>>> ????????????????????????????????????????????? ICU
>>> ?????????????????????????????????????????? "69.1"
>>> ????????????????????????????????????????????? TRE
>>> ??????????????????????? "TRE 0.8.0 R_fixes (BSD)"
>>> ??????????????????????????????????????????? iconv
>>> ???????????????????????????????????? "glibc 2.33"
>>> ???????????????????????????????????????? readline
>>> ??????????????????????????????????????????? "8.1"
>>> ???????????????????????????????????????????? BLAS
>>> "/home/cesko/r-devel/usr/lib64/R/lib/libRblas.so"
>>>
>>> Thanks for your observation that it works on your system - that
>>> implicates my readline-8.1 as being the culprit. Unfortunately, I
>>> don't dare attempt to downgrade it on my system to test, and
>>> regardless we still don't know why other readline-using
programs can
>>> paste in the same text with no issues.
>>>
>>>
>>> I've made some further progress on debugging: I noticed that
text
>>> <4096 bytes in size arrives fine (although sometimes with visual
>>> corruption), but text >4096 bytes doesn't. Pasting in the
result of
>>> perl -e 'print
("if(T)cat(\"a\")\n"x292)' works as expected, changing
>>> the 292 to 293 causes R to print a bunch of a's followed by the
>>> source code of the cat function.
>>>
>>>
>>> To still answer your question: with mgcv::gam, pasting in the first
>>> 94 lines (as printed by R with options(width=80)) produces a visual
>>> corruption of the prompt (it reads "G$family <-
>>> familyar.summaryintercept = drop.intercept))
control$scalePenalty,")
>>> but if I press return and type the closing "}" the code
has actually
>>> arrived just fine. The text up to and including that line is 4023
>>> bytes in size; when trying to add in more, it fails again.
>>>
>>>
>>> Cesko
>>>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>>
>>> *Van:* Tomas Kalibera <tomas.kalibera at gmail.com>
>>> *Verzonden:* zondag 13 juni 2021 10:00:27
>>> *Aan:* Voeten, C.C.; r-devel at r-project.org
>>> *Onderwerp:* Re: [Rd] Bracketed paste issues on Linux
>>> Thanks for the report. Could you please also post output from
>>> extSoftVersion() ?
>>>
>>> What happens if you paste just a smaller part of the code before
the
>>> long line? Is the output still corrupted? If so, is it corrupted
the
>>> same way, at the same places?
>>>
>>> (It seems to be working on my Ubuntu 20.04, readline 8.0, R-devel)
>>>
>>> Thanks
>>> Tomas
>>>
>>> On 6/12/21 3:44 PM, Cesko Voeten wrote:
>>> > I am on an up-to-date Arch Linux system, using the GNOME
desktop
>>> environment. By default, this turns on bracketed paste in terminal
>>> emulators; for those not familiar with this concept: it makes it so
>>> that if you paste in multiple lines of code, they are received in a
>>> single chunk. This works just fine with R, up to a certain amount
of
>>> text: for chunks past a certain length, some amount of text in the
>>> middle of the chunk goes missing. For example, if I print the
source
>>> of mgcv::gam into my R session and then attempt to copy and paste
it
>>> back in, what I end up with is:
>>> >
>>> > <snip 53 perfectly good lines>
>>> >????????????? pmf$formula <- gp$pf
>>> >????????????? pmf <- eval(pmf, parent.frame())
>>> > }?? objectvironment(attr(object$pred.formula,
"full")) <-
>>> .GlobalEnv<- environment(object$terms) <-
environment(object$pterms)
>>> <- .GlobalEnv
>>> >
>>> > So:
>>> >?? - the first 55 lines in this example arrive perfectly fine
>>> >?? - then a bunch go completely missing
>>> >?? - then various parts of the last few lines are jumbled
together
>>> into one line
>>> >
>>> > For reference on the third point, the actual last 10 lines of
my
>>> version of mgcv::gam are:
>>> >????? if (is.null(object$deviance))
>>> >????????? object$deviance <- sum(residuals(object,
"deviance")^2)
>>> >????? names(object$gcv.ubre) <- method
>>> >????? environment(object$formula) <-
>>> environment(object$pred.formula) <- environment(object$terms)
<-
>>> environment(object$pterms) <- .GlobalEnv
>>> >????? if (!is.null(object$model))
>>> >????????? environment(attr(object$model, "terms"))
<- .GlobalEnv
>>> >????? if (!is.null(attr(object$pred.formula,
"full")))
>>> >????????? environment(attr(object$pred.formula,
"full")) <- .GlobalEnv
>>> >????? object
>>> > }
>>> >
>>> > parts of which can be recognized in the last line of what was
pasted.
>>> > Naturally, the pasted function is not parsed properly: if I
press
>>> return I get the expected "+" signaling that the REPL is
expecting
>>> more input. So it is not merely a visual issue.
>>> >
>>> > I can reproduce this both in GNOME Terminal and in xterm, so
it is
>>> not a bug specific to my terminal emulator. In addition, pasting
the
>>> exact same code into either vim or nano running within the same
>>> terminal works fine. So I believe that this may be a bug in R
itself.
>>> It's easy to work around by disabling bracketed paste in the
>>> terminal, but it would be great if this could actually be made to
>>> work, especially given that bracketed paste is the default on my
>>> desktop environment.
>>> >
>>> > If given an account, I would be happy to file this as a bug;
let me
>>> know if that is desired. In the meantime, have others run into this
>>> and perhaps identified the root cause and/or a different
workaround?
>>> >
>>> > Thanks,
>>> > Cesko
>>> >
>>> > sessionInfo():
>>> >
>>> > R version 4.1.0 (2021-05-18)
>>> > Platform: x86_64-pc-linux-gnu (64-bit)
>>> > Running under: Arch Linux
>>> >
>>> > Matrix products: default
>>> > BLAS/LAPACK: /opt/intel/mkl/lib/intel64/libmkl_gf_lp64.so
>>> >
>>> > locale:
>>> >?? [1] LC_CTYPE=nl_NL.UTF-8?????? LC_NUMERIC=C
>>> >?? [3] LC_TIME=nl_NL.UTF-8??????? LC_COLLATE=nl_NL.UTF-8
>>> >?? [5] LC_MONETARY=nl_NL.UTF-8 LC_MESSAGES=nl_NL.UTF-8
>>> >?? [7] LC_PAPER=nl_NL.UTF-8?????? LC_NAME=C
>>> >?? [9] LC_ADDRESS=C?????????????? LC_TELEPHONE=C
>>> > [11] LC_MEASUREMENT=nl_NL.UTF-8 LC_IDENTIFICATION=C
>>> >
>>> > attached base packages:
>>> > [1] stats???? graphics? grDevices utils???? datasets methods??
base
>>> >
>>> > loaded via a namespace (and not attached):
>>> > [1] compiler_4.1.0? Matrix_1.3-4??? mgcv_1.8-36 splines_4.1.0
>>> > [5] nlme_3.1-152??? grid_4.1.0????? lattice_0.20-44
>>> >
>>> > ______________________________________________
>>> > R-devel at r-project.org mailing list
>>> > https://stat.ethz.ch/mailman/listinfo/r-devel
>>> <https://stat.ethz.ch/mailman/listinfo/r-devel>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Brian D. Ripley, ripley at stats.ox.ac.uk
Emeritus Professor of Applied Statistics, University of Oxford
One can also disable bracketed paste for R in .inputrc: $if R ? set enable-bracketed-paste off $endif Tomas On 6/15/21 11:09 AM, Prof Brian Ripley wrote:> I would have used source("clipboard") on systems which support it > (Tomas has confirmed it works on Linux).? See ?file. > > The macOS equivalent source(pipe("pbpaste")) also works. > > On 14/06/2021 11:06, Cesko Voeten wrote: >> Making it 1024 times larger gives: >> >> installing 'sysdata.rda' >> Error: segfault from C stack overflow >> >> Making it only 4 times larger provides a usable R. In my test case of >> copying&pasting mgcv::gam, I observe the same visual corruption at >> the prompt as before, but when pressing return it has actually been >> received correctly. My real-world problem involved a file 33KiB in >> size, which - as expected, since 16KiB < 33KiB - still has the same >> problem as before. >> >> I know nothing about readline, but I presume that there is no way for >> this buffer size to be dynamically resized at run time. In that case, >> maybe R should simply force-disable readline's bracketed paste? By >> the way, according to readline's changelog, this does indeed seem to >> be a feature that changed (viz. was enabled in more places) from >> readline-8.0 to readline-8.1. >> >> Finally, please disregard my earlier comment about vim and nano >> working just fine. They do, but they don't actually use readline >> (according to ldd), so don't provide a valid comparison. >> >> Thanks for your efforts! >> Cesko >> >> On 14-06-2021 at 08:33, Tomas Kalibera wrote: >>> Thanks, Cesko, for more debugging. As you are already compiling the >>> code, could you please try increasing CONSOLE_BUFFER_SIZE in >>> ./include/Defn.h from 4096 to some very large value (e.g. 1024 >>> times), rebuild R and check if the problems (not all bytes received >>> correctly, visual corruption) go away for texts of the size you >>> looked at before? >>> >>> >>> Thanks, >>> Tomas >>> >>> >>> >>> On 6/13/21 10:59 AM, Voeten, C.C. wrote: >>>> >>>> Thanks for looking into this! I've just compiled today's R-devel >>>> snapshot, and it shows the same issue. extSoftVersion() from that >>>> build: >>>> >>>> ???????????????????????????????????????????? zlib >>>> ???????????????????????????????????????? "1.2.11" >>>> ??????????????????????????????????????????? bzlib >>>> ???????????????????????????? "1.0.8, 13-Jul-2019" >>>> ?????????????????????????????????????????????? xz >>>> ????????????????????????????????????????? "5.2.5" >>>> ???????????????????????????????????????????? PCRE >>>> ?????????????????????????????? "10.37 2021-05-26" >>>> ????????????????????????????????????????????? ICU >>>> ?????????????????????????????????????????? "69.1" >>>> ????????????????????????????????????????????? TRE >>>> ??????????????????????? "TRE 0.8.0 R_fixes (BSD)" >>>> ??????????????????????????????????????????? iconv >>>> ???????????????????????????????????? "glibc 2.33" >>>> ???????????????????????????????????????? readline >>>> ??????????????????????????????????????????? "8.1" >>>> ???????????????????????????????????????????? BLAS >>>> "/home/cesko/r-devel/usr/lib64/R/lib/libRblas.so" >>>> >>>> Thanks for your observation that it works on your system - that >>>> implicates my readline-8.1 as being the culprit. Unfortunately, I >>>> don't dare attempt to downgrade it on my system to test, and >>>> regardless we still don't know why other readline-using programs >>>> can paste in the same text with no issues. >>>> >>>> >>>> I've made some further progress on debugging: I noticed that text >>>> <4096 bytes in size arrives fine (although sometimes with visual >>>> corruption), but text >4096 bytes doesn't. Pasting in the result of >>>> perl -e 'print ("if(T)cat(\"a\")\n"x292)' works as expected, >>>> changing the 292 to 293 causes R to print a bunch of a's followed >>>> by the source code of the cat function. >>>> >>>> >>>> To still answer your question: with mgcv::gam, pasting in the first >>>> 94 lines (as printed by R with options(width=80)) produces a visual >>>> corruption of the prompt (it reads "G$family <- >>>> familyar.summaryintercept = drop.intercept)) >>>> control$scalePenalty,") but if I press return and type the closing >>>> "}" the code has actually arrived just fine. The text up to and >>>> including that line is 4023 bytes in size; when trying to add in >>>> more, it fails again. >>>> >>>> >>>> Cesko >>>> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ >>>> >>>> *Van:* Tomas Kalibera <tomas.kalibera at gmail.com> >>>> *Verzonden:* zondag 13 juni 2021 10:00:27 >>>> *Aan:* Voeten, C.C.; r-devel at r-project.org >>>> *Onderwerp:* Re: [Rd] Bracketed paste issues on Linux >>>> Thanks for the report. Could you please also post output from >>>> extSoftVersion() ? >>>> >>>> What happens if you paste just a smaller part of the code before the >>>> long line? Is the output still corrupted? If so, is it corrupted the >>>> same way, at the same places? >>>> >>>> (It seems to be working on my Ubuntu 20.04, readline 8.0, R-devel) >>>> >>>> Thanks >>>> Tomas >>>> >>>> On 6/12/21 3:44 PM, Cesko Voeten wrote: >>>> > I am on an up-to-date Arch Linux system, using the GNOME desktop >>>> environment. By default, this turns on bracketed paste in terminal >>>> emulators; for those not familiar with this concept: it makes it so >>>> that if you paste in multiple lines of code, they are received in a >>>> single chunk. This works just fine with R, up to a certain amount >>>> of text: for chunks past a certain length, some amount of text in >>>> the middle of the chunk goes missing. For example, if I print the >>>> source of mgcv::gam into my R session and then attempt to copy and >>>> paste it back in, what I end up with is: >>>> > >>>> > <snip 53 perfectly good lines> >>>> >????????????? pmf$formula <- gp$pf >>>> >????????????? pmf <- eval(pmf, parent.frame()) >>>> > }?? objectvironment(attr(object$pred.formula, "full")) <- >>>> .GlobalEnv<- environment(object$terms) <- >>>> environment(object$pterms) <- .GlobalEnv >>>> > >>>> > So: >>>> >?? - the first 55 lines in this example arrive perfectly fine >>>> >?? - then a bunch go completely missing >>>> >?? - then various parts of the last few lines are jumbled together >>>> into one line >>>> > >>>> > For reference on the third point, the actual last 10 lines of my >>>> version of mgcv::gam are: >>>> >????? if (is.null(object$deviance)) >>>> >????????? object$deviance <- sum(residuals(object, "deviance")^2) >>>> >????? names(object$gcv.ubre) <- method >>>> >????? environment(object$formula) <- >>>> environment(object$pred.formula) <- environment(object$terms) <- >>>> environment(object$pterms) <- .GlobalEnv >>>> >????? if (!is.null(object$model)) >>>> >????????? environment(attr(object$model, "terms")) <- .GlobalEnv >>>> >????? if (!is.null(attr(object$pred.formula, "full"))) >>>> >????????? environment(attr(object$pred.formula, "full")) <- >>>> .GlobalEnv >>>> >????? object >>>> > } >>>> > >>>> > parts of which can be recognized in the last line of what was >>>> pasted. >>>> > Naturally, the pasted function is not parsed properly: if I press >>>> return I get the expected "+" signaling that the REPL is expecting >>>> more input. So it is not merely a visual issue. >>>> > >>>> > I can reproduce this both in GNOME Terminal and in xterm, so it >>>> is not a bug specific to my terminal emulator. In addition, pasting >>>> the exact same code into either vim or nano running within the same >>>> terminal works fine. So I believe that this may be a bug in R >>>> itself. It's easy to work around by disabling bracketed paste in >>>> the terminal, but it would be great if this could actually be made >>>> to work, especially given that bracketed paste is the default on my >>>> desktop environment. >>>> > >>>> > If given an account, I would be happy to file this as a bug; let >>>> me know if that is desired. In the meantime, have others run into >>>> this and perhaps identified the root cause and/or a different >>>> workaround? >>>> > >>>> > Thanks, >>>> > Cesko >>>> > >>>> > sessionInfo(): >>>> > >>>> > R version 4.1.0 (2021-05-18) >>>> > Platform: x86_64-pc-linux-gnu (64-bit) >>>> > Running under: Arch Linux >>>> > >>>> > Matrix products: default >>>> > BLAS/LAPACK: /opt/intel/mkl/lib/intel64/libmkl_gf_lp64.so >>>> > >>>> > locale: >>>> >?? [1] LC_CTYPE=nl_NL.UTF-8?????? LC_NUMERIC=C >>>> >?? [3] LC_TIME=nl_NL.UTF-8??????? LC_COLLATE=nl_NL.UTF-8 >>>> >?? [5] LC_MONETARY=nl_NL.UTF-8 LC_MESSAGES=nl_NL.UTF-8 >>>> >?? [7] LC_PAPER=nl_NL.UTF-8?????? LC_NAME=C >>>> >?? [9] LC_ADDRESS=C?????????????? LC_TELEPHONE=C >>>> > [11] LC_MEASUREMENT=nl_NL.UTF-8 LC_IDENTIFICATION=C >>>> > >>>> > attached base packages: >>>> > [1] stats???? graphics? grDevices utils???? datasets methods?? base >>>> > >>>> > loaded via a namespace (and not attached): >>>> > [1] compiler_4.1.0? Matrix_1.3-4??? mgcv_1.8-36 splines_4.1.0 >>>> > [5] nlme_3.1-152??? grid_4.1.0????? lattice_0.20-44 >>>> > >>>> > ______________________________________________ >>>> > R-devel at r-project.org mailing list >>>> > https://stat.ethz.ch/mailman/listinfo/r-devel >>>> <https://stat.ethz.ch/mailman/listinfo/r-devel> >> ______________________________________________ >> R-devel at r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel > >