similar to: Is it possible to increase MAX_NUM_DLLS in future R releases?

On 04/05/2016 08:44, Martin Maechler wrote: >>>>>> Qin Zhu <qinzhu at outlook.com> >>>>>> on Mon, 2 May 2016 16:19:44 -0400 writes: > > > Hi, > > I?m working on a Shiny app for statistical analysis. I ran into this "maximal number of DLLs reached" issue recently because my app requires importing many other packages.

Thanks for all your great answers. The app I?m working on is indeed an exploratory data analysis tool for gene expression, which requires a bunch of bioconductor packages. I guess for now, my best solution is to divide my app into modules and load/unload packages as the user switch from one module to another. This brought me another question: it seems that unload package with the

>>>>> Qin Zhu <qinzhu at outlook.com> >>>>> on Mon, 2 May 2016 16:19:44 -0400 writes: > Hi, > I?m working on a Shiny app for statistical analysis. I ran into this "maximal number of DLLs reached" issue recently because my app requires importing many other packages. > I?ve posted my question on stackoverflow

>>>>> Qin Zhu <qinzhu at outlook.com> >>>>> on Fri, 6 May 2016 11:33:37 -0400 writes: > Thanks for all your great answers. > The app I?m working on is indeed an exploratory data analysis tool for gene expression, which requires a bunch of bioconductor packages. > I guess for now, my best solution is to divide my app into modules and

On 05/04/2016 05:15 AM, Prof Brian Ripley wrote: > On 04/05/2016 08:44, Martin Maechler wrote: >>>>>>> Qin Zhu <qinzhu at outlook.com> >>>>>>> on Mon, 2 May 2016 16:19:44 -0400 writes: >> >> > Hi, >> > I?m working on a Shiny app for statistical analysis. I ran into >> this "maximal number of DLLs

I have read both the historical r-devel threads and the most recent one regarding this. After reviewing the code, it would seem to beg for a linked-list implementation vs. an array implementation. This has several interesting consequences. Cons: 1) We pay for dynamic allocation every time we register a DLL. This isn't a problem in the grand scheme of things as registering a DLL results in

This is a request to increase MAX_NUM_DLLS in Rdynload.c in from 100 to 500. On line 131 of Rdynload.c, changing #define MAX_NUM_DLLS 100 to #define MAX_NUM_DLLS 500 In development of the mlr package, there have been several episodes in the past where we have had to break up unit tests because of the "maximum number of DLLs reached" error. This error has been an inconvenience that

On Tue, Dec 20, 2016 at 7:04 AM, Henrik Bengtsson <henrik.bengtsson at gmail.com> wrote: > On reason for hitting the MAX_NUM_DLLS (= 100) limit is because some > packages don't unload their DLLs when they being unloaded themselves. I am surprised by this. Why does R not do this automatically? What is the case for keeping the DLL loaded after the package has been unloaded? What

On Tue, Dec 20, 2016 at 7:39 AM, Karl Millar <kmillar at google.com> wrote: > It's not always clear when it's safe to remove the DLL. > > The main problem that I'm aware of is that native objects with > finalizers might still exist (created by R_RegisterCFinalizer etc). > Even if there are no live references to such objects (which would be > hard to verify), it

Thanks Henrik this is very helpful! I will try this out on our tests and see if gcDLLs() has a positive effect. mlr currently has tests broken down by learner type such as classification, regression, forecasting, clustering, etc.. There are 83 classifiers alone so even when loading and unloading across learner types we can still hit the MAX_NUM_DLLS error, meaning we'll have to break them

On 20 December 2016 at 17:40, Martin Maechler wrote: | >>>>> Steve Bronder <sbronder at stevebronder.com> | >>>>> on Tue, 20 Dec 2016 01:34:31 -0500 writes: | | > Thanks Henrik this is very helpful! I will try this out on our tests and | > see if gcDLLs() has a positive effect. | | > mlr currently has tests broken down by learner type

On reason for hitting the MAX_NUM_DLLS (= 100) limit is because some packages don't unload their DLLs when they being unloaded themselves. In other words, there may be left-over DLLs just sitting there doing nothing but occupying space. You can remove these, using: R.utils::gcDLLs() Maybe that will help you get through your tests (as long as you're unloading packages). gcDLLs() will

>>>>> Steve Bronder <sbronder at stevebronder.com> >>>>> on Tue, 20 Dec 2016 01:34:31 -0500 writes: > Thanks Henrik this is very helpful! I will try this out on our tests and > see if gcDLLs() has a positive effect. > mlr currently has tests broken down by learner type such as classification, > regression, forecasting,

Hi, Dirk: On 12/20/2016 10:56 AM, Dirk Eddelbuettel wrote: > On 20 December 2016 at 17:40, Martin Maechler wrote: > | >>>>> Steve Bronder <sbronder at stevebronder.com> > | >>>>> on Tue, 20 Dec 2016 01:34:31 -0500 writes: > | > | > Thanks Henrik this is very helpful! I will try this out on our tests and > | > see if gcDLLs()

It does, but you'd still be relying on the R code ensuring that all of these objects are dead prior to unloading the DLL, otherwise they'll survive the GC. Maybe if the package counted how many such objects exist, it could work out when it's safe to remove the DLL. I'm not sure that it can be done automatically. What could be done is to to keep the DLL loaded, but remove it from

It's not always clear when it's safe to remove the DLL. The main problem that I'm aware of is that native objects with finalizers might still exist (created by R_RegisterCFinalizer etc). Even if there are no live references to such objects (which would be hard to verify), it still wouldn't be safe to unload the DLL until a full garbage collection has been done. If the DLL is

Hello, My problem is that I hit the hard-coded MAX_NUM_DLLS (100) limit of the number of loaded DLLs. I have a number of custom packages which interface and integrate a lot of CRAN and Bioconductor packages. For example, on my installation: Rscript -e 'library(crlmm);print(length(getLoadedDLLs()))' gives 28 loaded DLLs. I am currently trying to work-around that by putting external

Hi, I'm using dyn.load to load a shared library (compiled from C code) into R. If I dyn.unload it and then dyn.load it again, I get an hourglass icon in Rgui (R 1.7.0, Win 2000), and it just sits there forever. I can't press Escape to stop the current computation, but I can close Rgui without resorting to using the Task Manager. Is it a problem with my use of R_alloc? Do I need

Hello, I've been playing around w/ the new dynamic C symbol stuff (thank you for doing that!) in the, um, 5/11/01 R-devel package and, unless I've missed something incredibly obvious, there doesn't appear to be a mechanism for registering functions of an embedding executable. It seems like this would be a Good Thing to have. Since I'm doing most of my work in Windows at the

Hi, I'm trying to register my native routines using R_registerRoutines (...). I can compile the code, but the loader cannot resolve the symbol: undefined symbol: _Z18R_registerRoutinesP8_DllInfoPK12R_CMethodDefPK15R_CallMethodDefS3_S6 _ $ nm bgx.Rcheck/bgx/libs/bgx.so | grep R_registerRoutines U _Z18R_registerRoutinesP8_DllInfoPK12R_CMethodDefPK15R_CallMethodDefS3_S6