R help - Jul 2007 - Speeding up

Hi R-helpers,

I am new in programming and R, so this may be basic.  I need to speed up
a procedure where I minimize some function on different partitions of
the data. I can do this with a loop, as for instance in:

i<-1
meanmin<-Inf
while (i<length(x)) {
	ind<-x<xord[i]
	if (some function to be minimized<meanmin) {
		meanmin<-some function to be minimized
		indmin<-xord[i]
	}
i<-i+1
}
print(indmin)

I learned that I should avoid loops, so I found the following
alternative:

dmat<-outer(x,xord,"<")*1 
ss<-apply(dmat,2,function (z) some function to be minimized)
indmin<-xord[which.min(ss)]
print(indmin)

But this does not lead to spectacular improvements (obviously, this is
dependent on the function and the length of x, and this dmat does not
seem to be an elegant solution to me).  Is there scope for substantial
improvement? Any pointers will be greatly appreciated.  Below an example
with some timings.

> set.seed(12345)
> x <- rnorm(1000, mean = 5, sd = 2)
> xord<-x[order(x)]
> 
> system.time({i<-1
+ meanmin<-Inf
+ while (i<length(x)) {
+ ind<-x<xord[i]
+ if ((mean(x*ind)+mean(x*!ind))<meanmin) {
+ meanmin<-mean(x*ind)+mean(x*!ind)
+ indmin<-xord[i]
+ }
+ i<-i+1
+ }
+ print(indmin)})
[1] 3.826595
   user  system elapsed 
   0.14    0.00    0.14 
> 
> 
> 
> system.time({dmat<-outer(x,xord,"<")*1 
+ ss<-apply(dmat,2,function (z) mean(x*z)+mean(x*!z))
+ indmin<-xord[which.min(ss)]
+ print(indmin)})
[1] 3.826595
   user  system elapsed 
   0.42    0.06    0.49 
> 
>

Dear Bjorn,

Do you know that mean(xy*ind)+mean(xy*!ind) yields the same value for
all i? Maybe you meant mean(xy[ind]) + mean(xy[!ind])

sapply(xord, function(xordi, xy = x){
  ind <- xy < xordi
  mean(xy*ind)+mean(xy*!ind)
})

Cheers,

Thierry

------------------------------------------------------------------------
----
ir. Thierry Onkelinx
Instituut voor natuur- en bosonderzoek / Research Institute for Nature
and Forest
Cel biometrie, methodologie en kwaliteitszorg / Section biometrics,
methodology and quality assurance
Gaverstraat 4
9500 Geraardsbergen
Belgium
tel. + 32 54/436 185
Thierry.Onkelinx op inbo.be
www.inbo.be 

Do not put your faith in what statistics say until you have carefully
considered what they do not say.  ~William W. Watt
A statistical analysis, properly conducted, is a delicate dissection of
uncertainties, a surgery of suppositions. ~M.J.Moroney

 
> -----Oorspronkelijk bericht-----
> Van: r-help-bounces op stat.math.ethz.ch 
> [mailto:r-help-bounces op stat.math.ethz.ch] Namens Van Campenhout Bjorn
> Verzonden: maandag 9 juli 2007 15:12
> Aan: r-help op stat.math.ethz.ch
> Onderwerp: [R] Speeding up
> 
> Hi R-helpers,
> 
> I am new in programming and R, so this may be basic.  I need 
> to speed up a procedure where I minimize some function on 
> different partitions of the data. I can do this with a loop, 
> as for instance in:
> 
> i<-1
> meanmin<-Inf
> while (i<length(x)) {
> 	ind<-x<xord[i]
> 	if (some function to be minimized<meanmin) {
> 		meanmin<-some function to be minimized
> 		indmin<-xord[i]
> 	}
> i<-i+1
> }
> print(indmin)
> 
> I learned that I should avoid loops, so I found the following
> alternative:
> 
> dmat<-outer(x,xord,"<")*1
> ss<-apply(dmat,2,function (z) some function to be minimized) 
> indmin<-xord[which.min(ss)]
> print(indmin)
> 
> But this does not lead to spectacular improvements 
> (obviously, this is dependent on the function and the length 
> of x, and this dmat does not seem to be an elegant solution 
> to me).  Is there scope for substantial improvement? Any 
> pointers will be greatly appreciated.  Below an example with 
> some timings.
> 
> 
> > set.seed(12345)
> > x <- rnorm(1000, mean = 5, sd = 2)
> > xord<-x[order(x)]
> > 
> > system.time({i<-1
> + meanmin<-Inf
> + while (i<length(x)) {
> + ind<-x<xord[i]
> + if ((mean(x*ind)+mean(x*!ind))<meanmin) {
> + meanmin<-mean(x*ind)+mean(x*!ind)
> + indmin<-xord[i]
> + }
> + i<-i+1
> + }
> + print(indmin)})
> [1] 3.826595
>    user  system elapsed 
>    0.14    0.00    0.14 
> > 
> > 
> > 
> > system.time({dmat<-outer(x,xord,"<")*1
> + ss<-apply(dmat,2,function (z) mean(x*z)+mean(x*!z)) 
> + indmin<-xord[which.min(ss)]
> + print(indmin)})
> [1] 3.826595
>    user  system elapsed 
>    0.42    0.06    0.49 
> > 
> >
> 
> ______________________________________________
> R-help op stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide 
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>