Hi everyone, I'm thinking about breaking up a large package I have to avoid loading some of the more specialised parts when it isn't necessary. The package is dependent on shared libraries. The base one will keep all of the core stuff like constructors etc and the specialised parts will need to access these. It is obviously possible to set the "local" flag to false, however this becomes a dangerous, especially with the larger package. Is there any way to define a group of packages that share a symbol table? That would seem to me to be a possible solution. Can anyone offer advice on this? Thanks -- Richard Beare, CSIRO Mathematical & Information Sciences Locked Bag 17, North Ryde, NSW 1670, Australia Phone: +61-2-93253221 (GMT+~10hrs) Fax: +61-2-93253200 Richard.Beare@csiro.au -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-devel-request@stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
On Fri, 06 Sep 2002 18:23:33 +1000, you wrote:>The package is dependent on shared libraries. The base one >will keep all of the core stuff like constructors etc and >the specialised parts will need to access these. It is >obviously possible to set the "local" flag to false, however >this becomes a dangerous, especially with the larger >package. > >Is there any way to define a group of packages that share a >symbol table? That would seem to me to be a possible >solution. > >Can anyone offer advice on this?Are you talking about the R symbol table, or the symbol table in the dynamically linked libraries? If the former, then (until namespaces are implemented) they *always* share symbol tables. Every routine can see every other routine. Your comment about a "local" flag makes me think you're talking about an external library. This may be system dependent. In Windows it's not possible to do what you want to do. A DLL exports certain symbols, and there's no way to merge that table with the table from another DLL. However, it is normal to import routines specifying both the routine name and which DLL they come from, so name clashes don't need to be a problem. Duncan Murdoch -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-devel-request@stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Hi Richard.
This issue arises reasonably often. Doug Bates has brought it up on a
few occassions, specifically related to packages that want to use
routines that are located in R modules (lapack). And many of the
Omegahat packages are naturally split into different packages that
share a set of common routines.
As you point out, you can use
dyn.load("foo.so", local = FALSE)
to get the symbols in foo.so directly into
the global symbol table and make them available
to other dynamically loaded code.
This can introduce a whole range of other problems
There are a few different approaches you can use.
a) Put the common code into a native library (e.g. libCommon.a or libCommon.so)
and then arrange to have each of the packages that use these routines
link against this. For Windows, you will explicitly export the symbols
that are to be shared from this library.
In this model, the library of common routines (libCommon.so) is not
used dynamically loaded by R. Instead is one step removed, being
linked into .so's or .dll's that are.
This is probably the most practical approach. However, it will require
that you set your LD_LIBRARY_PATH (or PATH on Windows) variable
to include the directory containing libCommon.so/dll so that the dynamic loader
can find it at run time.
Also, if you use global variables in the common library, some systems
require coaxing to ensure that there is only one instance of those variables
shared across all the different libraries.
b) If you really want to have
I have some example packages that illustrate this. If there is
interest, I will tidy them up and write up a more complete description
of the different approaches.
Kurt Hornik and I have agreed to implement a mechanism after R 1.6 is
released. It will probably use one or more of these approaches and
make them much easier.
Richard.Beare@csiro.au wrote:> Hi everyone,
>
> I'm thinking about breaking up a large package I have to
> avoid loading some of the more specialised parts when it
> isn't necessary.
>
> The package is dependent on shared libraries. The base one
> will keep all of the core stuff like constructors etc and
> the specialised parts will need to access these. It is
> obviously possible to set the "local" flag to false, however
> this becomes a dangerous, especially with the larger
> package.
>
> Is there any way to define a group of packages that share a
> symbol table? That would seem to me to be a possible
> solution.
>
> Can anyone offer advice on this?
>
> Thanks
>
> --
> Richard Beare, CSIRO Mathematical & Information Sciences
> Locked Bag 17, North Ryde, NSW 1670, Australia
> Phone: +61-2-93253221 (GMT+~10hrs) Fax: +61-2-93253200
>
> Richard.Beare@csiro.au
>
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
> r-devel mailing list -- Read
http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> Send "info", "help", or "[un]subscribe"
> (in the "body", not the subject !) To:
r-devel-request@stat.math.ethz.ch
>
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
--
_______________________________________________________________
Duncan Temple Lang duncan@research.bell-labs.com
Bell Labs, Lucent Technologies office: (908)582-3217
700 Mountain Avenue, Room 2C-259 fax: (908)582-3340
Murray Hill, NJ 07974-2070
http://cm.bell-labs.com/stat/duncan
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !) To:
r-devel-request@stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Oops. Thought I was postponing that mail and sent it instead.
So here is the alternative approach, i.e. b)
Rather than use the linker to connect the calls to the common routines
and their definitions in a secondary library (e.g. libCommon.so), you
can resolve and set the symbols yourself. Specifically, you use
function pointers in the code that calls the shared routines and then
when you load the packages, you arrange to have these variables refer
to the actual routines that you want.
For example, suppose we have two packages - A and B and A has a
routine named foo() that we want to call in the routine bar() located
in B.
So A.c has
void
foo(void) {
...
}
And B.c has
void (*foo_ptr)(void);
void
bar()
{
/* Want to call bar() */
(*foo_ptr)();
}
Given this setup, you need to have an initialization
routine in B.c that sets the value of the variable foo_ptr.
There are two ways to do this.
You can call R_FindSymbol() within C code.
Using the initialization routine in package B
void R_init_B(DllInfo *dll)
{
foo_ptr = R_FindSymbol("foo", "A", NULL);
}
Alternatively, you can do the lookup in R and pass
the address of foo() as an argument to a routine in B.
a <- getNativeSymbolInfo("foo", PACKAGE = "A")
.Call("setFoo", a, PACKAGE = "B")
And the C routine setFoo() looks something like
SEXP setFoo(SEXP address)
{
foo_ptr = R_ExternalPtrAddr(address);
return(R_NilValue);
}
This approach, compared to linking libraries, allows you to
dynamically change routines. This can be useful but is probably
overkill for your example. However, this approach does not rely
on linkers idiosyncrasies.
Hope this helps.
D.
--
_______________________________________________________________
Duncan Temple Lang duncan@research.bell-labs.com
Bell Labs, Lucent Technologies office: (908)582-3217
700 Mountain Avenue, Room 2C-259 fax: (908)582-3340
Murray Hill, NJ 07974-2070
http://cm.bell-labs.com/stat/duncan
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !) To:
r-devel-request@stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._