Hi all, When S4 methods are defined on base function (say, "match"), the function becomes a method with the body "base::match(x,y)". A call to such a function often spends more time doing "::" than in the function itself. I always assumed that "::" was a very low-level thing, but it turns out to be a plain old function defined in base/R/namespace.R. What would you all think about making "::" and ":::" .Primitives? I have submitted some examples, timings, and a patch to the R bug tracker (https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=16134). I'd be very interested to hear your thoughts on the matter. Regards, Pete ____________________ Peter M. Haverty, Ph.D. Genentech, Inc. phaverty at gene.com
I'm not convinced that how to make :: faster is the right question. If you are finding foo::bar being called often enough to matter to your overall performance then to me the question is: why are you calling foo::bar more than once? Making :: a bit faster by making it a primitive will remove some overhead, but your are still left with a lot of work that shouldn't need to happen more than once. For default methods there ought to be a way to create those so the default method is computed at creation or load time and stored in an environment. For other cases if I want to use foo::bar many times, say in a loop, I would do foo_bar <- foo::bar and use foo_bar, or something along those lines. When :: and ::: were introduce they were intended primarily for reflection and debugging, so speed was not an issue. ::: is still really only reliably usable that way, and making it faster may just encourage bad practice. :: is different and there are good arguments for using it in code, but I'm not yet seeing good arguments for use in ways that would be performance-critical, but I'm happy to be convinced otherwise. If there is a need for a faster :: then going to a SPECIALSXP is fine; it would also be good to make the byte code compiler aware of it, and possibly to work on ways to improve the performance further e.g. through cacheing. Best, luke On Thu, 22 Jan 2015, Peter Haverty wrote:> Hi all, > > When S4 methods are defined on base function (say, "match"), the > function becomes a method with the body "base::match(x,y)". A call to > such a function often spends more time doing "::" than in the function > itself. I always assumed that "::" was a very low-level thing, but it > turns out to be a plain old function defined in base/R/namespace.R. > What would you all think about making "::" and ":::" .Primitives? I > have submitted some examples, timings, and a patch to the R bug > tracker (https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=16134). > I'd be very interested to hear your thoughts on the matter. > > Regards, > Pete > > ____________________ > Peter M. Haverty, Ph.D. > Genentech, Inc. > phaverty at gene.com > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel >-- Luke Tierney Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics and Fax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: luke-tierney at uiowa.edu Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu
On Thu, Jan 22, 2015 at 1:44 PM, <luke-tierney at uiowa.edu> wrote:> I'm not convinced that how to make :: faster is the right question. If > you are finding foo::bar being called often enough to matter to your > overall performance then to me the question is: why are you calling > foo::bar more than once? Making :: a bit faster by making it a > primitive will remove some overhead, but your are still left with a > lot of work that shouldn't need to happen more than once. > > For default methods there ought to be a way to create those so the > default method is computed at creation or load time and stored in an > environment. For other cases if I want to use foo::bar many times, say > in a loop, I would do > > foo_bar <- foo::bar > > and use foo_bar, or something along those lines. > > When :: and ::: were introduce they were intended primarily for > reflection and debugging, so speed was not an issue. ::: is still > really only reliably usable that way, and making it faster may just > encourage bad practice. :: is different and there are good arguments > for using it in code, but I'm not yet seeing good arguments for use in > ways that would be performance-critical, but I'm happy to be convinced > otherwise. If there is a need for a faster :: then going to a > SPECIALSXP is fine; it would also be good to make the byte code > compiler aware of it, and possibly to work on ways to improve the > performance further e.g. through cacheing. >I think you will find that no matter how much it does not matter in terms of performance, folks will avoid :: out of principle if they think its slower. We're conditioned to write efficient code even when it does not really impact real world usage. As using :: is good practice in many contexts, making it fast will encourage folks to use it. THK> > Best, > > luke > > > On Thu, 22 Jan 2015, Peter Haverty wrote: > > > Hi all, >> >> When S4 methods are defined on base function (say, "match"), the >> function becomes a method with the body "base::match(x,y)". A call to >> such a function often spends more time doing "::" than in the function >> itself. I always assumed that "::" was a very low-level thing, but it >> turns out to be a plain old function defined in base/R/namespace.R. >> What would you all think about making "::" and ":::" .Primitives? I >> have submitted some examples, timings, and a patch to the R bug >> tracker (https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=16134). >> I'd be very interested to hear your thoughts on the matter. >> >> Regards, >> Pete >> >> ____________________ >> Peter M. Haverty, Ph.D. >> Genentech, Inc. >> phaverty at gene.com >> >> ______________________________________________ >> R-devel at r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel >> >> > -- > Luke Tierney > Ralph E. Wareham Professor of Mathematical Sciences > University of Iowa Phone: 319-335-3386 > Department of Statistics and Fax: 319-335-3017 > Actuarial Science > 241 Schaeffer Hall email: luke-tierney at uiowa.edu > Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu > > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel >-- http://www.keittlab.org/ [[alternative HTML version deleted]]
On Thu, Jan 22, 2015 at 11:44 AM, <luke-tierney at uiowa.edu> wrote:> I'm not convinced that how to make :: faster is the right question. If > you are finding foo::bar being called often enough to matter to your > overall performance then to me the question is: why are you calling > foo::bar more than once? Making :: a bit faster by making it a > primitive will remove some overhead, but your are still left with a > lot of work that shouldn't need to happen more than once. > > For default methods there ought to be a way to create those so the > default method is computed at creation or load time and stored in an > environment. For other cases if I want to use foo::bar many times, say > in a loop, I would do > > foo_bar <- foo::bar > > and use foo_bar, or something along those lines.While you're on the line: Do you think this is an optimization that the 'compiler' package and it's cmpfun() byte compiler will be able to do in the future? /Henrik> > When :: and ::: were introduce they were intended primarily for > reflection and debugging, so speed was not an issue. ::: is still > really only reliably usable that way, and making it faster may just > encourage bad practice. :: is different and there are good arguments > for using it in code, but I'm not yet seeing good arguments for use in > ways that would be performance-critical, but I'm happy to be convinced > otherwise. If there is a need for a faster :: then going to a > SPECIALSXP is fine; it would also be good to make the byte code > compiler aware of it, and possibly to work on ways to improve the > performance further e.g. through cacheing. > > Best, > > luke > > > On Thu, 22 Jan 2015, Peter Haverty wrote: > > >> Hi all, >> >> When S4 methods are defined on base function (say, "match"), the >> function becomes a method with the body "base::match(x,y)". A call to >> such a function often spends more time doing "::" than in the function >> itself. I always assumed that "::" was a very low-level thing, but it >> turns out to be a plain old function defined in base/R/namespace.R. >> What would you all think about making "::" and ":::" .Primitives? I >> have submitted some examples, timings, and a patch to the R bug >> tracker (https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=16134). >> I'd be very interested to hear your thoughts on the matter. >> >> Regards, >> Pete >> >> ____________________ >> Peter M. Haverty, Ph.D. >> Genentech, Inc. >> phaverty at gene.com >> >> ______________________________________________ >> R-devel at r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel >> > > -- > Luke Tierney > Ralph E. Wareham Professor of Mathematical Sciences > University of Iowa Phone: 319-335-3386 > Department of Statistics and Fax: 319-335-3017 > Actuarial Science > 241 Schaeffer Hall email: luke-tierney at uiowa.edu > Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu > > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel
On Thu, Jan 22, 2015 at 11:44 AM, <luke-tierney at uiowa.edu> wrote:> > For default methods there ought to be a way to create those so the > default method is computed at creation or load time and stored in an > environment.We had considered that, but we thought the definition of the function would be easier to interpret if it explicitly specified the namespace, instead of using tricks with environments. The same applies for memoizing the lookup in front of a loop. The implementation of these functions is almost simpler in C than it is in R, so there is relatively little risk to this change. But I agree the benefits are also somewhat minor.> For other cases if I want to use foo::bar many times, say > in a loop, I would do > > foo_bar <- foo::bar > > and use foo_bar, or something along those lines. > > When :: and ::: were introduce they were intended primarily for > reflection and debugging, so speed was not an issue. ::: is still > really only reliably usable that way, and making it faster may just > encourage bad practice. :: is different and there are good arguments > for using it in code, but I'm not yet seeing good arguments for use in > ways that would be performance-critical, but I'm happy to be convinced > otherwise. If there is a need for a faster :: then going to a > SPECIALSXP is fine; it would also be good to make the byte code > compiler aware of it, and possibly to work on ways to improve the > performance further e.g. through cacheing. > > Best, > > luke > > > On Thu, 22 Jan 2015, Peter Haverty wrote: > > >> Hi all, >> >> When S4 methods are defined on base function (say, "match"), the >> function becomes a method with the body "base::match(x,y)". A call to >> such a function often spends more time doing "::" than in the function >> itself. I always assumed that "::" was a very low-level thing, but it >> turns out to be a plain old function defined in base/R/namespace.R. >> What would you all think about making "::" and ":::" .Primitives? I >> have submitted some examples, timings, and a patch to the R bug >> tracker (https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=16134). >> I'd be very interested to hear your thoughts on the matter. >> >> Regards, >> Pete >> >> ____________________ >> Peter M. Haverty, Ph.D. >> Genentech, Inc. >> phaverty at gene.com >> >> ______________________________________________ >> R-devel at r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel >> > > -- > Luke Tierney > Ralph E. Wareham Professor of Mathematical Sciences > University of Iowa Phone: 319-335-3386 > Department of Statistics and Fax: 319-335-3017 > Actuarial Science > 241 Schaeffer Hall email: luke-tierney at uiowa.edu > Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu > > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel
> if I want to use foo::bar many times, say > in a loop, I would do > > foo_bar <- foo::bar > > and use foo_bar, or something along those lines.The foreach package does that with a function from the compiler package, so that foreach can work on old version of R: comp <- if (getRversion() < "2.13.0") { function(expr, ...) expr } else { compiler::compile } This results in foreach having its own copy of compiler::compile, with namespace "compiler", but copied from the version of package:compile existing on the machine that built the binary of foreach. If you later install an updated version of the compiler package, then foreach still uses the old compiler::compile, which may not work with the private functions in the new version of package:compiler. Making :: faster would not fix this particular problem (making 'comp' a function that contained the if(getRVersion...) code would), but things like this could cause problems when more people put 'myFunc <- otherPackage::Func' in their packages. Bill Dunlap TIBCO Software wdunlap tibco.com On Thu, Jan 22, 2015 at 11:44 AM, <luke-tierney at uiowa.edu> wrote:> I'm not convinced that how to make :: faster is the right question. If > you are finding foo::bar being called often enough to matter to your > overall performance then to me the question is: why are you calling > foo::bar more than once? Making :: a bit faster by making it a > primitive will remove some overhead, but your are still left with a > lot of work that shouldn't need to happen more than once. > > For default methods there ought to be a way to create those so the > default method is computed at creation or load time and stored in an > environment. For other cases if I want to use foo::bar many times, say > in a loop, I would do > > foo_bar <- foo::bar > > and use foo_bar, or something along those lines. > > When :: and ::: were introduce they were intended primarily for > reflection and debugging, so speed was not an issue. ::: is still > really only reliably usable that way, and making it faster may just > encourage bad practice. :: is different and there are good arguments > for using it in code, but I'm not yet seeing good arguments for use in > ways that would be performance-critical, but I'm happy to be convinced > otherwise. If there is a need for a faster :: then going to a > SPECIALSXP is fine; it would also be good to make the byte code > compiler aware of it, and possibly to work on ways to improve the > performance further e.g. through cacheing. > > Best, > > luke > > > On Thu, 22 Jan 2015, Peter Haverty wrote: > > > Hi all, >> >> When S4 methods are defined on base function (say, "match"), the >> function becomes a method with the body "base::match(x,y)". A call to >> such a function often spends more time doing "::" than in the function >> itself. I always assumed that "::" was a very low-level thing, but it >> turns out to be a plain old function defined in base/R/namespace.R. >> What would you all think about making "::" and ":::" .Primitives? I >> have submitted some examples, timings, and a patch to the R bug >> tracker (https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=16134). >> I'd be very interested to hear your thoughts on the matter. >> >> Regards, >> Pete >> >> ____________________ >> Peter M. Haverty, Ph.D. >> Genentech, Inc. >> phaverty at gene.com >> >> ______________________________________________ >> R-devel at r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel >> >> > -- > Luke Tierney > Ralph E. Wareham Professor of Mathematical Sciences > University of Iowa Phone: 319-335-3386 > Department of Statistics and Fax: 319-335-3017 > Actuarial Science > 241 Schaeffer Hall email: luke-tierney at uiowa.edu > Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu > > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel >[[alternative HTML version deleted]]