R devel - Apr 2015 - RObjectTables freezes in R 3.2.0 RC on 32bit systems

We recently started noticing freezes that appear only on 32bit systems
(both linux and windows) with a relatively recent versions of R 3.2.0,
including the RC. It looks like the problem can be traced back to the use
of R_ObjectTables (see R_ext/Callbacks.h)

The problem is a bit difficult to reproduce because it does not appear on
x64 and because the official R interface to this functionality, the
RObjectTables package from OmegaHat, is no longer maintained. I created a
slightly updated version of RObjectTables package (added a NAMESPACE file)
to reproduce the problem:

  devtools::install_github("jeroenooms/RObjectTables")

The problem can be triggered using the included example from the
DirectoryObjectTable manual.

  library(RObjectTables)
  db <- DirectoryObjectTable(tempdir())
  dbwrite(db, "x", 1:10)
  dbwrite(db, "y", letters[1:3])
  dbobjects(db)
  dbread(db, "x")
  attach(newRFunctionTable(db), name = "myRData")

Things work as expected up till dbread(), but once the object-table is
attached, R freezes on 32bit whereas it works as expected on 64bit.

The same problem appears for other packages that are calling the object
tables C interfaces directly without using the RObjectTables R package.

	[[alternative HTML version deleted]]

On Tue, Apr 14, 2015 at 6:29 PM, Jeroen Ooms <jeroenooms at gmail.com>
wrote:
> Things work as expected up till dbread(), but once the object-table is
> attached, R freezes on 32bit whereas it works as expected on 64bit.
Debugging this some more, it looks like the freeze appears at the very
end of the attach function, when it calls length(names(value)) on the
newly created object-tables environment. At this stage, calling
ls(value) gives the expected output, but calling names(value) or
length(value) triggers the freeze.

The NEWS file does mention that R 3.2.0 introduces some changes to
names(env) internals. Could it be that this somehow conflicts with the
object tables hooks?

Thanks for the report.
I'll take a look at this, but it will take a week or so before I have time.

Best,
 Duncan

On 4/14/15 6:29 PM, Jeroen Ooms wrote:
> We recently started noticing freezes that appear only on 32bit systems
> (both linux and windows) with a relatively recent versions of R 3.2.0,
> including the RC. It looks like the problem can be traced back to the use
> of R_ObjectTables (see R_ext/Callbacks.h)
> 
> The problem is a bit difficult to reproduce because it does not appear on
> x64 and because the official R interface to this functionality, the
> RObjectTables package from OmegaHat, is no longer maintained. I created a
> slightly updated version of RObjectTables package (added a NAMESPACE file)
> to reproduce the problem:
> 
>   devtools::install_github("jeroenooms/RObjectTables")
> 
> The problem can be triggered using the included example from the
> DirectoryObjectTable manual.
> 
>   library(RObjectTables)
>   db <- DirectoryObjectTable(tempdir())
>   dbwrite(db, "x", 1:10)
>   dbwrite(db, "y", letters[1:3])
>   dbobjects(db)
>   dbread(db, "x")
>   attach(newRFunctionTable(db), name = "myRData")
> 
> Things work as expected up till dbread(), but once the object-table is
> attached, R freezes on 32bit whereas it works as expected on 64bit.
> 
> The same problem appears for other packages that are calling the object
> tables C interfaces directly without using the RObjectTables R package.
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
> 
-- 
Director, Data Sciences Initiative, UC Davis
Professor, Dept. of Statistics, UC Davis

http://datascience.ucdavis.edu
http://www.stat.ucdavis.edu/~duncan

>>>>> Jeroen Ooms <jeroenooms at gmail.com>
>>>>>     on Tue, 14 Apr 2015 23:11:01 -0700 writes:
    > On Tue, Apr 14, 2015 at 6:29 PM, Jeroen Ooms
    > <jeroenooms at gmail.com> wrote:
    >> Things work as expected up till dbread(), but once the
    >> object-table is attached, R freezes on 32bit whereas it
    >> works as expected on 64bit.

    > Debugging this some more, it looks like the freeze appears
    > at the very end of the attach function, when it calls
    > length(names(value)) on the newly created object-tables
    > environment. At this stage, calling ls(value) gives the
    > expected output, but calling names(value) or length(value)
    > triggers the freeze.

Well, attach() itself has been modified since R 3.1.x; it now
does use  
     length(names(value))    where  value <- .Internal(attach(..))

where as before it used

    length(ls(envir = value, all.names = TRUE))

and the new code is considerably faster for regular cases,
but as you say is not correctly working for the special
RobjectsTables stuff.

Maybe this helps to find a last minute patch to that part in the
R 3.2.0 RC sources ?

    > The NEWS file does mention that R 3.2.0 introduces some
    > changes to names(env) internals. Could it be that this
    > somehow conflicts with the object tables hooks?

    > ______________________________________________
    > R-devel at r-project.org mailing list
    > https://stat.ethz.ch/mailman/listinfo/r-devel

Looks like names() never worked on object tables on any platfom; the
failure was just more obvious on 32-bit systems than on 64-bit ones.
This is now fixed in R-devel and R-patched.

Best,

luke

On Wed, 15 Apr 2015, Jeroen Ooms wrote:
> On Tue, Apr 14, 2015 at 6:29 PM, Jeroen Ooms <jeroenooms at
gmail.com> wrote:
>> Things work as expected up till dbread(), but once the object-table is
>> attached, R freezes on 32bit whereas it works as expected on 64bit.
>
> Debugging this some more, it looks like the freeze appears at the very
> end of the attach function, when it calls length(names(value)) on the
> newly created object-tables environment. At this stage, calling
> ls(value) gives the expected output, but calling names(value) or
> length(value) triggers the freeze.
>
> The NEWS file does mention that R 3.2.0 introduces some changes to
> names(env) internals. Could it be that this somehow conflicts with the
> object tables hooks?
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:   luke-tierney at uiowa.edu
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu