R devel - Dec 2013 - SQLite database file gets corrupted and XP goes blue

Dear fellows,

I'm getting blue screen on XP while populating SQLite database, using
RStudio and RQLite package.

The blue screen says "INVALID PROCESS ATTACH ATTEMPT" before goes into
memory check and reboot.
The database (file) is corrupted and not readable after this happens.

I'm iterating over CSV files, each contans a time-series of as many as
500x5000: rows being dates, columns being dependent variables.  For each
CSV creates a 500x5000 matrix (preallocated), and it is "remove"-ed
immediately after its use, among other relatively large objects.

Without this explicit resource deallocation, the blue screen stuff
occurred at much earlier during the process.

Still, XP's Process Monitor shows steady increase in memory usage. 

My XP has 3GB RAM.  It seems that the screen goes blue when the usage
goes near 1GB.

The database file, when it is corrupted, has only 150MB.  SQLite's max
db size is 2TB, XP32's max file size is 4GB.  So, I'm well under the
limits.

In the past, when an R process goes out of memory, it rather graceful
returned, like "Out of memory".

I have closed any other applications that accesses the database such as
SQLite Browser.

Any insight appreciated.
Sachko

I'd recommend sending a reproducible example to the maintainer of the
RSQLite package (cc'd in case he doesn't follow r-devel any more)
Hadley

On Mon, Dec 30, 2013 at 7:06 PM, Sachko Honda
<sachko.honda at mountainpacificgroup.com> wrote:
> Dear fellows,
>
> I'm getting blue screen on XP while populating SQLite database, using
> RStudio and RQLite package.
>
> The blue screen says "INVALID PROCESS ATTACH ATTEMPT" before goes
into
> memory check and reboot.
> The database (file) is corrupted and not readable after this happens.
>
> I'm iterating over CSV files, each contans a time-series of as many as
> 500x5000: rows being dates, columns being dependent variables.  For each
> CSV creates a 500x5000 matrix (preallocated), and it is
"remove"-ed
> immediately after its use, among other relatively large objects.
>
> Without this explicit resource deallocation, the blue screen stuff
> occurred at much earlier during the process.
>
> Still, XP's Process Monitor shows steady increase in memory usage.
>
> My XP has 3GB RAM.  It seems that the screen goes blue when the usage
> goes near 1GB.
>
> The database file, when it is corrupted, has only 150MB.  SQLite's max
> db size is 2TB, XP32's max file size is 4GB.  So, I'm well under
the limits.
>
> In the past, when an R process goes out of memory, it rather graceful
> returned, like "Out of memory".
>
> I have closed any other applications that accesses the database such as
> SQLite Browser.
>
> Any insight appreciated.
> Sachko
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel


-- 
http://had.co.nz/