search for: gcdlls

Thanks Henrik this is very helpful! I will try this out on our tests and see if gcDLLs() has a positive effect. mlr currently has tests broken down by learner type such as classification, regression, forecasting, clustering, etc.. There are 83 classifiers alone so even when loading and unloading across learner types we can still hit the MAX_NUM_DLLS error, meaning we'll have to...

...20 December 2016 at 17:40, Martin Maechler wrote: | >>>>> Steve Bronder <sbronder at stevebronder.com> | >>>>> on Tue, 20 Dec 2016 01:34:31 -0500 writes: | | > Thanks Henrik this is very helpful! I will try this out on our tests and | > see if gcDLLs() has a positive effect. | | > mlr currently has tests broken down by learner type such as classification, | > regression, forecasting, clustering, etc.. There are 83 classifiers alone | > so even when loading and unloading across learner types we can still hit | > the...

This is a request to increase MAX_NUM_DLLS in Rdynload.c in from 100 to 500. On line 131 of Rdynload.c, changing #define MAX_NUM_DLLS 100 to #define MAX_NUM_DLLS 500 In development of the mlr package, there have been several episodes in the past where we have had to break up unit tests because of the "maximum number of DLLs reached" error. This error has been an inconvenience that

>>>>> Steve Bronder <sbronder at stevebronder.com> >>>>> on Tue, 20 Dec 2016 01:34:31 -0500 writes: > Thanks Henrik this is very helpful! I will try this out on our tests and > see if gcDLLs() has a positive effect. > mlr currently has tests broken down by learner type such as classification, > regression, forecasting, clustering, etc.. There are 83 classifiers alone > so even when loading and unloading across learner types we can still hit > the MAX_NUM_DL...

...40, Martin Maechler wrote: > | >>>>> Steve Bronder <sbronder at stevebronder.com> > | >>>>> on Tue, 20 Dec 2016 01:34:31 -0500 writes: > | > | > Thanks Henrik this is very helpful! I will try this out on our tests and > | > see if gcDLLs() has a positive effect. > | > | > mlr currently has tests broken down by learner type such as classification, > | > regression, forecasting, clustering, etc.. There are 83 classifiers alone > | > so even when loading and unloading across learner types we can still...

On reason for hitting the MAX_NUM_DLLS (= 100) limit is because some packages don't unload their DLLs when they being unloaded themselves. In other words, there may be left-over DLLs just sitting there doing nothing but occupying space. You can remove these, using: R.utils::gcDLLs() Maybe that will help you get through your tests (as long as you're unloading packages). gcDLLs() will look at base::getLoadedDLLs() and its content and compare to loadedNamespaces() and unregister any "stray" DLLs that remain after corresponding packages have been unloaded. I thi...

...tain crash when such objects get garbage collected. Very good point. Does base::gc() perform such a *full* garbage collection and thereby trigger all remaining finalizers to be called? In other words, do you think an explicit call to base::gc() prior to cleaning out left-over DLLs (e.g. R.utils::gcDLLs()) would be sufficient? /Henrik > > A better approach would be to just remove the limit on the number of > DLLs, dynamically expanding the array if/when needed. > > > On Tue, Dec 20, 2016 at 3:40 AM, Jeroen Ooms <jeroen.ooms at stat.ucla.edu> wrote: >> On Tue, Dec 20...

...garbage collected. > > Very good point. > > Does base::gc() perform such a *full* garbage collection and thereby > trigger all remaining finalizers to be called? In other words, do you > think an explicit call to base::gc() prior to cleaning out left-over > DLLs (e.g. R.utils::gcDLLs()) would be sufficient? > > /Henrik > >> >> A better approach would be to just remove the limit on the number of >> DLLs, dynamically expanding the array if/when needed. >> >> >> On Tue, Dec 20, 2016 at 3:40 AM, Jeroen Ooms <jeroen.ooms at stat.ucla.e...

On Tue, Dec 20, 2016 at 7:04 AM, Henrik Bengtsson <henrik.bengtsson at gmail.com> wrote: > On reason for hitting the MAX_NUM_DLLS (= 100) limit is because some > packages don't unload their DLLs when they being unloaded themselves. I am surprised by this. Why does R not do this automatically? What is the case for keeping the DLL loaded after the package has been unloaded? What