Displaying 20 results from an estimated 12000 matches similar to: "readLines on open connection reads only first write on MacOS"
2017 Sep 02
1
readLines() segfaults on large file & question on how to work around
Thank you for your suggestion. Unfortunately, while R doesn't segfault
calling readr::read_file() on the test file I described, I get the error
message:
Error in read_file_(ds, locale) : negative length vectors are not allowed
Jen
On Sat, Sep 2, 2017 at 1:38 PM, Ista Zahn <istazahn at gmail.com> wrote:
> As s work-around I suggest readr::read_file.
>
> --Ista
>
>
>
2017 Sep 02
5
readLines() segfaults on large file & question on how to work around
Hi:
I have a 2.1GB JSON file. Typically I use readLines() and
jsonlite:fromJSON() to extract data from a JSON file.
When I try and read in this file using readLines() R segfaults.
I believe the two salient issues with this file are
1). Its size
2). It is a single line (no line breaks)
I can reproduce this issue as follows
#Generate a big file with no line breaks
# In R
>
2017 Sep 02
0
readLines() segfaults on large file & question on how to work around
As s work-around I suggest readr::read_file.
--Ista
On Sep 2, 2017 2:58 PM, "Jennifer Lyon" <jennifer.s.lyon at gmail.com> wrote:
> Hi:
>
> I have a 2.1GB JSON file. Typically I use readLines() and
> jsonlite:fromJSON() to extract data from a JSON file.
>
> When I try and read in this file using readLines() R segfaults.
>
> I believe the two salient
2017 Sep 04
0
readLines() segfaults on large file & question on how to work around
Although the problem can apparently be avoided in this case. readLines
causing a segfault still seems unwanted behaviour to me. I can replicate
this with the example below (sessionInfo is further down):
# Generate an example file
l <- paste0(sample(c(letters, LETTERS), 1E6, replace = TRUE),
collapse="")
con <- file("test.txt", "wt")
for (i in
2017 Sep 03
2
readLines() segfaults on large file & question on how to work around
Jeroen:
Thank you for pointing me to ndjson, which I had not heard of and is
exactly my case.
My experience:
jsonlite::stream_in - segfaults
ndjson::stream_in - my fault, I am running Ubuntu 14.04 and it is too old
so it won't compile the package
corpus::read_ndjson - works!!! Of course it does a different simplification
than jsonlite::fromJSON, so I have to change some code, but
2014 Nov 13
4
[PATCH 0/4 v3] readline escaping functions
Helper functions for future support of backslash escaped spaces in filenames.
There are a few tests too. Changed according to review remarks and fixed few
other mistakes.
Maros Zatko (4):
fish: copy parse_quoted_string and hexdigit from fish.h to rl.c
fish: rl.{c,h} - escaping functions for readline
fish: basic tests for readline escaping
autotools: add fish/test
Makefile.am
2014 Nov 07
3
[PATCH 0/3] v2 readline escaping functions
From: Maros Zatko <mzatko@redhat.com>
Helper functions for future support of backslash escaped spaces in filenames.
There are a few tests too. Changed according to review remarks.
Maros Zatko (3):
fish: rl.{c, h} - escaping functions for readline
fish: basic tests for readline escaping
autotools: add fish/test
Makefile.am | 1 +
configure.ac | 1 +
2014 Oct 31
6
[PATCH 0/3] WIP readline escaping functions
From: Maros Zatko <hacxman@gmail.com>
Auxiliary functions for readline to support space character escaping
in filenames in future.
Escaping function is taken from fish.c (used to be parse_quoted_string)
plus its un-escaping counterpart. There are a few tests for both.
Maros Zatko (3):
fish: rl.{c,h} - escaping functions for readline
fish: basic tests for readline escaping
autotools:
2010 May 05
2
readLines with space-delimiter?
Hi,
I am reading a large space-delimited text file into R (41 columns and many
rows) and need to do run each row's values through another R object and then
write to another text file. So, far using readLines and writeLines seems to
be the best bet. I've gotten the data exchange working except each row is
read in as one 'chunk', meaning the row has all values between two quotes
2006 Feb 28
1
Typos in writeLines.Rd, readLines.Rd, and data.matrix.Rd
Hello,
The diffs below are based on revision 37445 and show
some typo corrections for writeLines.Rd, readLines.Rd,
and data.matrix.Rd that I'd like to bring to the list's
attention.
Sincerely,
Stephen Weigand
Rochester, Minnesota, USA
--- ./src/library/base/man/writeLines.Rd Sun Feb 26 13:46:06 2006
+++ /tmp/writeLines.Rd Sun Feb 26 20:53:44 2006
@@ -14,8 +14,8 @@
each
2019 Jul 02
1
eliminate a partial argument match warning in R CMD check
Hello,
I'm seeing a nuisance warning when I run `R CMD check --as-cran
whatever_x.y.z.tar.gz`.
I generally work with these options set:
options(
warnPartialMatchArgs = TRUE,
warnPartialMatchAttr = TRUE,
warnPartialMatchDollar = TRUE
)
And I see this:
* checking use of SHLIB_OPENMP_*FLAGS in Makefiles ...Warning in dir("src",
patt = "[.]c$") :
partial argument
2009 Sep 14
2
problems with reshape
Hello *
I would like to reshape wide the following dataset:
> rl <- read.dta("intermedi/rapporti_lavoro.dta") [c("id_rl","prog","sil_pi","sil_cf","sil_dat_avv")]
> dim(rl)
[1] 12964 5
> object.size(rl)
1194728 bytes
> head(rl)
id_rl prog sil_pi sil_cf sil_dat_avv
1 638 1 04567XXXXXX
2010 Jan 21
2
Problems completely reading in a "large" sized data set
I have been through the help file archives a number of times, and still
cannot figure out what is wrong.
I have a tab-delimited text file. 76Mb, so while it's large.. it's not
-that- large. I'm running Win7 x64 w/4G RAM and R 2.10.1
When I open this data in Excel, i have 27 rows and 450932 rows, excluding
the first row containing variable names.
I am trying to get this into R as a
2010 Jan 14
2
Fixed size permutations
I'm using functions to return a matrix of all permutations of a
specified size from a larger list for predictor selection.
For each predictor size I use a seperate function like this:
bag2 <- function(n) {
rl <- c()
for (i1 in seq(n)) {
for (i2 in seq(n)) {
if (length(unique(c(i1,i2)))==1) {next}
rl <- cbind(rl,matrix(c(i1,i2)))
}
}
2018 Feb 17
2
readLines interaction with gsub different in R-dev
I was told to re-raise this issue with R-dev:
In the documentation of R-dev and R-3.4.3, under ?gsub
> replacement
> ... For perl = TRUE only, it can also contain "\U" or "\L" to convert the rest of the replacement to upper or lower case and "\E" to end case conversion.
However, the following code runs differently:
tempf <- tempfile()
2013 Apr 10
1
Issue with Control-Z in a text file on Windows - readLines() appears to truncate
Working on Windows I have had to deal with CSV files that,
unfortunately, contain embedded Control-Zs, i.e. ASCII character 26 in
decimal, and the readLines() function in R on Windows (2.15.2 and
3.0.0) appears to truncate at the control-Z. There is no problem at
all on Ubuntu Linux with R 3.0.0.
Am I mistaken or is this genuine?
# Create a small file with embedded Control-Z
h3 <-
2018 Feb 17
0
readLines interaction with gsub different in R-dev
I think the problem in R-devel happens when there are non-ASCII characters
in any
of the strings passed to gsub.
txt <- vapply(list(as.raw(c(0x41, 0x6d, 0xc3, 0xa9, 0x6c, 0x69, 0x65)),
as.raw(c(0x41, 0x6d, 0x65, 0x6c, 0x69, 0x61))), rawToChar, "")
txt
#[1] "Am?lie" "Amelia"
Encoding(txt)
#[1] "unknown" "unknown"
gsub(perl=TRUE,
2018 Feb 17
2
readLines interaction with gsub different in R-dev
| Confirmed for R-devel (current) on Ubuntu 17.10. But ... isn't the regexp
| you use wrong, ie isn't R-devel giving the correct answer?
No, I don't think R-devel is correct (or at least consistent with the
documentation). My interpretation of gsub("(\\w)", "\\U\\1", entry,
perl = TRUE) is "Take every word character and replace it with itself,
converted to
2005 Mar 17
1
Binding one column of characters into a dataframe factors other numeric columns
Hi all,
I searched through the archives, but couldn't find a fix...
Basically, I've got a bunch of numeric vectors and one character vector that
I want to bind into a data frame. When I include the character vector as a
column in the data frame, all the numeric columns get factored in the data
frame, which makes it tough to call those columns for calculations later on.
I've tried
2013 Jun 11
1
Help needed in feature extraction from two input files
Hi,
Try this:
lines1<- readLines(textConnection("gene1 or1|1234 or3|56 or4|793
gene4 or2|347
gene5 or3|23 or7|123456789"))
lines2<-readLines(textConnection(">or1|1234
ATCGGATTCAGG
>or2|347
GAACCTATCGGGGGGGGAATTTATATATTTTA
>or3|56
ATCGGAGATATAACCAATC
>or3|23
AAAATTAACAAGAGAATAGACAAAAAAA
>or4|793
ATCTCTCTCCTCTCTCTCTAAAAA
>or7|123456789