julian.bothe at elitepartner.de
2014-Oct-20 10:26 UTC
[R] dbHasCompleted() always returns TRUE for POSTGRES Database with RJDBC?
Hello Simon, Hello everyone, I have a problem with retrieving database-queries chunkwise. In my query (Postgresql-Database, Postgres-Version 9.1) dbHasCompleted seems to always return TRUE. dbGetQuery works for smaller tables, but not for the whole query, since then a Heap-Overflow-Error will occur (and a Error: "java.lang.OutOfMemoryError: GC overhead limit exceeded" when trying to set a larger size). Does anybody else have or had this problem? All the best Julian ######## adapted Example from help (?dbHasCompleted ), Works ####### require("RSQLite") con <- dbConnect(RSQLite::SQLite(), ":memory:") dbWriteTable(con, "mtcars", mtcars) # Fetch all results res <- dbSendQuery(con, "SELECT * FROM mtcars WHERE cyl = 4") dbFetch(res) dbClearResult(res) # Fetch in chunks res <- dbSendQuery(con, "SELECT * FROM mtcars") dbHasCompleted(res) ## RETURNS FALSE !!! while (!dbHasCompleted(res)) { chunk <- fetch(res, 10) print(nrow(chunk)) } dbClearResult(res) dbDisconnect(con) ########## My Code, lamentably not reproducible because of needed Postgres-Server, not working ######### # Connect to Postgres-Database res2 <- dbSendQuery(Postgres_con, "select id from mytable limit 1000;" ) dbHasCompleted(res2) ##RETURNS TRUE! while (!dbHasCompleted(res2)) { chunk <- fetch(res2, 10 ) rbind(dk, chunk) } try({ dbClearResult(res2) rm(chunk,res2) }) [[alternative HTML version deleted]]