Displaying 6 results from an estimated 6 matches for "stream_in".
2017 Sep 03
2
readLines() segfaults on large file & question on how to work around
Jeroen:
Thank you for pointing me to ndjson, which I had not heard of and is
exactly my case.
My experience:
jsonlite::stream_in - segfaults
ndjson::stream_in - my fault, I am running Ubuntu 14.04 and it is too old
so it won't compile the package
corpus::read_ndjson - works!!! Of course it does a different simplification
than jsonlite::fromJSON, so I have to change some code, but it works
beautifully at l...
2017 Sep 04
0
readLines() segfaults on large file & question on how to work around
...piler_3.4.1 R6_2.2.2 hms_0.3 tools_3.4.1
tibble_1.3.3 Rcpp_0.12.12 rlang_0.1.2
On 03-09-17 20:50, Jennifer Lyon wrote:
> Jeroen:
>
> Thank you for pointing me to ndjson, which I had not heard of and is
> exactly my case.
>
> My experience:
> jsonlite::stream_in - segfaults
> ndjson::stream_in - my fault, I am running Ubuntu 14.04 and it is too old
> so it won't compile the package
> corpus::read_ndjson - works!!! Of course it does a different simplification
> than jsonlite::fromJSON, so I have to change some code, but it works...
2017 Sep 02
5
readLines() segfaults on large file & question on how to work around
Hi:
I have a 2.1GB JSON file. Typically I use readLines() and
jsonlite:fromJSON() to extract data from a JSON file.
When I try and read in this file using readLines() R segfaults.
I believe the two salient issues with this file are
1). Its size
2). It is a single line (no line breaks)
I can reproduce this issue as follows
#Generate a big file with no line breaks
# In R
>
2004 Jun 18
5
Patch to stop vcut from generating broken streams
Skipped content of type multipart/mixed-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://westfish.xiph.org/pipermail/vorbis-dev/attachments/20040618/e7bee431/attachment.pgp
2016 Oct 01
0
A few new packages on CRAN
- ndjdon : Wicked Fast ndjson Reader
Reads in ndjson significantly faster than jsonlite::stream_in(), flattens
each
JSON record and returns a data.table.
https://cran.r-project.org/web/packages/ndjson/index.html
- htmltidy : Clean Up or Pretty Print Gnarly HTML and XHTML
C-backed package that includes the HTML Tidy library. Useful for cleaning
up
HTML beyond what you get with the HTM...
2016 Oct 01
0
A few new packages on CRAN
- ndjdon : Wicked Fast ndjson Reader
Reads in ndjson significantly faster than jsonlite::stream_in(), flattens
each
JSON record and returns a data.table.
https://cran.r-project.org/web/packages/ndjson/index.html
- htmltidy : Clean Up or Pretty Print Gnarly HTML and XHTML
C-backed package that includes the HTML Tidy library. Useful for cleaning
up
HTML beyond what you get with the HTM...