search for: ndjon

...l.com> > wrote: > > I have a 2.1GB JSON file. Typically I use readLines() and > > jsonlite:fromJSON() to extract data from a JSON file. > > If your data consists of one json object per line, this is called > 'ndjson'. There are several packages specialized to read ndjon files: > > - corpus::read_ndjson > - ndjson::stream_in > - jsonlite::stream_in > > In particular the 'corpus' package handles large files really well > because it has an option to memory-map the file instead of reading all > of its data into memory. > > If t...

Hi: I have a 2.1GB JSON file. Typically I use readLines() and jsonlite:fromJSON() to extract data from a JSON file. When I try and read in this file using readLines() R segfaults. I believe the two salient issues with this file are 1). Its size 2). It is a single line (no line breaks) I can reproduce this issue as follows #Generate a big file with no line breaks # In R >

...te: >>> I have a 2.1GB JSON file. Typically I use readLines() and >>> jsonlite:fromJSON() to extract data from a JSON file. >> >> If your data consists of one json object per line, this is called >> 'ndjson'. There are several packages specialized to read ndjon files: >> >> - corpus::read_ndjson >> - ndjson::stream_in >> - jsonlite::stream_in >> >> In particular the 'corpus' package handles large files really well >> because it has an option to memory-map the file instead of reading all >> of i...