All, I am using a sum of squared differences in the objective function of an optimization problem I am doing and I have managed to speed it up using the outer function versus the nested for loops, but my suspicion is that the calculation could be done even quicker. Please see the code below for a simple example. If anyone can point out a faster way I would appreciate it greatly. Thanks, Roger X <- runif(1000) now <- proc.time() ans <- 0 for (i in 1:length(X)) { for (j in 1:length(X)) { ans <- ans + (X[i]-X[j])*(X[i]-X[j]) } } ans speed <- proc.time() - now; cat(" That took ", round(speed[3],1), " secs.\n", sep="") now <- proc.time() gg <- outer(X, X, FUN="-") sum(gg*gg) speed <- proc.time() - now; cat(" That took ", round(speed[3],1), " secs.\n", sep="")
Bos, Roger <roger.bos <at> rothschild.com> writes:> I am using a sum of squared differences in theobjective function of an optimization problem I am doing and I> have managed to speed it up using theouter function versus the nested for loops, but my suspicion is that> the calculation could be done even quicker.Please see the code below for a simple example. If anyone can> point out a faster way I would appreciate it greatly. > > Thanks, > > Roger > > X <- runif(1000) > > now <- proc.time() > ans <- 0 > for (i in 1:length(X)) { > for (j in 1:length(X)) { > ans <- ans + (X[i]-X[j])*(X[i]-X[j]) > } > } > ans > speed <- proc.time() - now; cat(" That took ",round(speed[3],1), " secs.\n", sep="")> > now <- proc.time() > gg <- outer(X, X, FUN="-") > sum(gg*gg) > speed <- proc.time() - now; cat(" That took ",round(speed[3],1), " secs.\n", sep="")> >system.time( for (i in 1:length(X)) { for (j in 1:length(X)) { ans <- ans + (X[i]-X[j])*(X[i]-X[j]) } }) user system elapsed 2.241 0.009 2.293 system.time(2 * sum(c(dist(X))^2)) user system elapsed 0.038 0.002 0.040 and then there is Rcpp if you want to add some extra grease. -- Kenneth Knoblauch Inserm U846 Stem-cell and Brain Research Institute Department of Integrative Neurosciences 18 avenue du Doyen L?pine 69500 Bron France tel: +33 (0)4 72 91 34 77 fax: +33 (0)4 72 91 34 61 portable: +33 (0)6 84 10 64 10 http://www.sbri.fr/members/kenneth-knoblauch.html
> I am using a sum of squared differences in the objective function of an > optimization problem I am doing and I have managed to speed it up using > the outer function versus the nested for loops, but my suspicion is > that the calculation could be done even quicker. Please see the code > below for a simple example. If anyone can point out a faster way I > would appreciate it greatly.First, look at ?system.time ; it's intended for finding out how much compute time a command takes. You're also calculating elapsed time which is dependent on (among other things) other processes running so maybe not the most reliable thing for benchmarking. Second, outer() is fast because it avoids loops and uses the fastest vector routines R can (essentially, replicating the vector and then relying on simple subtraction of vectors). In principle calling outer() loses a little time in internal checks on what object types and function you've provided etc.; in practice that doesn't add up to much. You can check by using just the core code from outer with an added multiplication instead of the dimensioning statement: ssqdif <- function(X, Y=X) { #From 'outer' without modification Y <- rep(Y, rep.int(length(X), length(Y))) X <- rep(X, times = ceiling(length(Y)/length(X))) #For this case: sum((X-Y)^2) #SLIGHTLY quicker than d<-X-Y; sum(d*d) } system.time(ssqdif(X)) #Comparing outer() method: system.time({gg <- outer(X, X, FUN="-"); sum(gg*gg)}) There's little practical difference; both hover from 0.00 to 0.03 s system time. I could barely tell the difference even averaged over 100 runs; I was getting an average around 0.007 (system time) and 2.5s user time for both methods. Conclusion: hard to beat outer() for this purpose in R S Ellison ******************************************************************* This email and any attachments are confidential. Any use...{{dropped:8}}
Hi Patric, Thanks for your explanation and suggestions. I didn't know that it might happen. Sorry about that. A good lesson for the future post. Thanks again. Halim On Wed, 23 Oct 2013 21:00:28 +1300, Patrick Connolly wrote> This is a request to anyone who starts a new subject to begin with a > new message and NOT reply to an existing one. If your mail client is > any good, it's very simple to set up an alias (mine is simply 'r') so > that the tedious task of typing 'r-help at stat.math.ethz.ch' is > unnecessary and it's quicker than scrolling through an address book. > It's also quicker than deleting the previous subject. > > Most mornings, I have as many as a hundred messages mostly from R- > help and it's very useful to have them threaded. However, the usefulness > of threading is lost when posters reply to a message and then change > the subject instead of creating a new message. > > People who don't have a mail client that can display email in threads > are probably unaware that this sort of thing can happen in ones that > do: > > 37 N 25 Jan Luis Silva ( 34) [R] plot/screen > 38 N 25 Jan Uwe Ligges ( 55) `-> > 39 N 25 Jan Fernando Henrique Ferra ( 20) [R] Plotting > coloured histograms -> 40 N 26 Jan Mohamed A. Kerasha ( 12) > |->[R] Distributions. 41 N 26 Jan ripley at stats.ox.ac.uk ( 26) > | |-> 42 26 Jan Qin Xin ( 9) | `->[R] how > could I add legends 43 27 Jan Ko-Kang Kevin Wang ( 31) | > `-> 44 N 26 Jan Remigijus Lapinskas ( 32) |->Re: [R] > Plotting coloured his 45 N 26 Jan Damon Wischik (125) > `-> 46 N 25 Jan Rex_Bryan at urscorp.com ( 10) [R] plotting > primatives, ellipse 47 N 25 Jan Uwe Ligges ( 19) `- > > > > As Martin Maechler explained some time ago, it also screws up the > archives for a similar reason. What's more, you run the risk of > having your message deleted if it's in a thread that is being deleted > for a reason quite unrelated to your question, so fewer people might > see it. > > Your cooperation will be greatly appreciated. > > -- > ~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.___ Patrick Connolly {~._.~} Great minds discuss ideas _( Y )_ Average minds discuss events> (:_~*~_:) Small minds discuss people > (_)-(_) ..... Eleanor Roosevelt > > ~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~. > > -- > This message has been scanned for viruses and > dangerous content by MailScanner, and is > believed to be clean.--------------- Md. Abdul Halim Assistant Professor Department of Forestry and Environmental Science Shahjalal University of Science and Technology,Sylhet-3114, Bangladesh. Cell: +8801714078386. alt. e-mail: xou03 at yahoo.com -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
Seemingly Similar Threads
- grDevices::convertColor XYZ space is it really xyY?
- Recoding variables in R
- Instalando R en Linux Debian
- how to adjust link function in logistic regression to predict the proportion of correct responses in 2AFC task?
- Fw: nested linear model; with common intercept