On 23/09/2018 3:31 PM, Jeff Newmiller wrote: [lots of good stuff deleted]> Vectorize is > syntactic sugar with a performance penalty.[More deletions.] I would say Vectorize isn't just "syntactic sugar". When I use that term, I mean something that looks nice but is functionally equivalent. However, Vectorize() really does something useful: some functions (e.g. outer()) take other functions as arguments, but they assume the argument is a vectorized function. If it is not, they fail, or generate garbage results. Vectorize() is designed to modify the interface to a function so it acts as if it is vectorized. The "performance penalty" part of your statement is true. It will generally save some computing cycles to write a new function using a for loop instead of using Vectorize(). But that may waste some programmer time. Duncan Murdoch (writing as one of the authors of Vectorize()) P.S. I'd give an example of syntactic sugar, but I don't want to bruise some other author's feelings :-).
Very insightful. Thanks, Duncan Based on your opinion, is there any benefit to use the parallelism in the corporate computing environment where the size of data is far more than million rows and there are multiple cores in the server. Actually the practice of going concurrency or not is more related to my production tasks instead of something academic. Really appreciate your thoughts. On Sun, Sep 23, 2018 at 2:42 PM Duncan Murdoch <murdoch.duncan at gmail.com> wrote:> On 23/09/2018 3:31 PM, Jeff Newmiller wrote: > > [lots of good stuff deleted] > > > Vectorize is > > syntactic sugar with a performance penalty. > > [More deletions.] > > I would say Vectorize isn't just "syntactic sugar". When I use that > term, I mean something that looks nice but is functionally equivalent. > > However, Vectorize() really does something useful: some functions (e.g. > outer()) take other functions as arguments, but they assume the argument > is a vectorized function. If it is not, they fail, or generate garbage > results. Vectorize() is designed to modify the interface to a function > so it acts as if it is vectorized. > > The "performance penalty" part of your statement is true. It will > generally save some computing cycles to write a new function using a for > loop instead of using Vectorize(). But that may waste some programmer > time. > > Duncan Murdoch > (writing as one of the authors of Vectorize()) > > P.S. I'd give an example of syntactic sugar, but I don't want to bruise > some other author's feelings :-). >[[alternative HTML version deleted]]
On Sun, 23 Sep 2018, Duncan Murdoch wrote:> On 23/09/2018 3:31 PM, Jeff Newmiller wrote: > > [lots of good stuff deleted] > >> Vectorize is >> syntactic sugar with a performance penalty. > > [More deletions.] > > I would say Vectorize isn't just "syntactic sugar". When I use that term, I > mean something that looks nice but is functionally equivalent. > > However, Vectorize() really does something useful: some functions (e.g. > outer()) take other functions as arguments, but they assume the argument is a > vectorized function. If it is not, they fail, or generate garbage results. > Vectorize() is designed to modify the interface to a function so it acts as > if it is vectorized. > > The "performance penalty" part of your statement is true. It will generally > save some computing cycles to write a new function using a for loop instead > of using Vectorize(). But that may waste some programmer time. > > Duncan Murdoch > (writing as one of the authors of Vectorize()) > > P.S. I'd give an example of syntactic sugar, but I don't want to bruise some > other author's feelings :-).Perhaps my writing needs some syntactic sugar: inefficient looping algorithms can make sense when the calculations performed in each iteration are long and/or involve large amounts of data. As I mentioned earlier in this thread I use for loops fairly often, and I use other inefficient syntactic sugar as well but only to organize large blocks of already-vectorized (lowercase) calculation units. In addition to the potential for inefficient use of programmer time, vectorizing code increases the maximum amount of memory used during execution of your program. A for loop is one simple way to allow memory re-use so really large problems can be solved with limited resources, and some syntactic sugar such as Vectorize can make it easier to keep track of those for loops. --------------------------------------------------------------------------- Jeff Newmiller The ..... ..... Go Live... DCN:<jdnewmil at dcn.davis.ca.us> Basics: ##.#. ##.#. Live Go... Live: OO#.. Dead: OO#.. Playing Research Engineer (Solar/Batteries O.O#. #.O#. with /Software/Embedded Controllers) .OO#. .OO#. rocks...1k
On 23/09/2018 4:00 PM, Wensui Liu wrote:> Very insightful. Thanks, Duncan > > Based on your opinion, is there any benefit to use the parallelism in > the corporate computing environment where the size of data is far more > than million rows and there are multiple cores in the server.I would say "try it and see". Sometimes it probably helps a lot, sometimes it's probably detrimental. Duncan Murdoch P.S. I last worked in a corporate computing environment 40 years ago when I was still wet behind the ears, so you'd probably want to ask someone else. However, more recently I worked in an academic environment where I learned to say "try it and see" in many different ways. You just got the basic one today.> > Actually the practice of going concurrency or not is more related to my > production tasks instead of something academic. > > Really appreciate your thoughts. > > On Sun, Sep 23, 2018 at 2:42 PM Duncan Murdoch <murdoch.duncan at gmail.com > <mailto:murdoch.duncan at gmail.com>> wrote: > > On 23/09/2018 3:31 PM, Jeff Newmiller wrote: > > [lots of good stuff deleted] > > > Vectorize is > > syntactic sugar with a performance penalty. > > [More deletions.] > > I would say Vectorize isn't just "syntactic sugar".? When I use that > term, I mean something that looks nice but is functionally equivalent. > > However, Vectorize() really does something useful:? some functions > (e.g. > outer()) take other functions as arguments, but they assume the > argument > is a vectorized function.? If it is not, they fail, or generate garbage > results.? Vectorize() is designed to modify the interface to a function > so it acts as if it is vectorized. > > The "performance penalty" part of your statement is true.? It will > generally save some computing cycles to write a new function using a > for > loop instead of using Vectorize().? But that may waste some > programmer time. > > Duncan Murdoch > (writing as one of the authors of Vectorize()) > > P.S. I'd give an example of syntactic sugar, but I don't want to bruise > some other author's feelings :-). >