On Tue, Dec 20, 2016 at 7:04 AM, Henrik Bengtsson <henrik.bengtsson at gmail.com> wrote:> On reason for hitting the MAX_NUM_DLLS (= 100) limit is because some > packages don't unload their DLLs when they being unloaded themselves.I am surprised by this. Why does R not do this automatically? What is the case for keeping the DLL loaded after the package has been unloaded? What happens if you reload another version of the same package from a different library after unloading?
It's not always clear when it's safe to remove the DLL. The main problem that I'm aware of is that native objects with finalizers might still exist (created by R_RegisterCFinalizer etc). Even if there are no live references to such objects (which would be hard to verify), it still wouldn't be safe to unload the DLL until a full garbage collection has been done. If the DLL is unloaded, then the function pointer that was registered now becomes a pointer into the memory where the DLL was, leading to an almost certain crash when such objects get garbage collected. A better approach would be to just remove the limit on the number of DLLs, dynamically expanding the array if/when needed. On Tue, Dec 20, 2016 at 3:40 AM, Jeroen Ooms <jeroen.ooms at stat.ucla.edu> wrote:> On Tue, Dec 20, 2016 at 7:04 AM, Henrik Bengtsson > <henrik.bengtsson at gmail.com> wrote: >> On reason for hitting the MAX_NUM_DLLS (= 100) limit is because some >> packages don't unload their DLLs when they being unloaded themselves. > > I am surprised by this. Why does R not do this automatically? What is > the case for keeping the DLL loaded after the package has been > unloaded? What happens if you reload another version of the same > package from a different library after unloading? > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel
Henrik Bengtsson
2016-Dec-21 17:10 UTC
[Rd] Request: Increasing MAX_NUM_DLLS in Rdynload.c
On Tue, Dec 20, 2016 at 7:39 AM, Karl Millar <kmillar at google.com> wrote:> It's not always clear when it's safe to remove the DLL. > > The main problem that I'm aware of is that native objects with > finalizers might still exist (created by R_RegisterCFinalizer etc). > Even if there are no live references to such objects (which would be > hard to verify), it still wouldn't be safe to unload the DLL until a > full garbage collection has been done. > > If the DLL is unloaded, then the function pointer that was registered > now becomes a pointer into the memory where the DLL was, leading to an > almost certain crash when such objects get garbage collected.Very good point. Does base::gc() perform such a *full* garbage collection and thereby trigger all remaining finalizers to be called? In other words, do you think an explicit call to base::gc() prior to cleaning out left-over DLLs (e.g. R.utils::gcDLLs()) would be sufficient? /Henrik> > A better approach would be to just remove the limit on the number of > DLLs, dynamically expanding the array if/when needed. > > > On Tue, Dec 20, 2016 at 3:40 AM, Jeroen Ooms <jeroen.ooms at stat.ucla.edu> wrote: >> On Tue, Dec 20, 2016 at 7:04 AM, Henrik Bengtsson >> <henrik.bengtsson at gmail.com> wrote: >>> On reason for hitting the MAX_NUM_DLLS (= 100) limit is because some >>> packages don't unload their DLLs when they being unloaded themselves. >> >> I am surprised by this. Why does R not do this automatically? What is >> the case for keeping the DLL loaded after the package has been >> unloaded? What happens if you reload another version of the same >> package from a different library after unloading? >> >> ______________________________________________ >> R-devel at r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel