This is some kind of follow-up to my previous posts. I have further
improved the speed of my program 6 times by eliminating all the
apply(). It turns out that apply is slow, is slower than direct loop,
it is an order slower than a matrix operation alternative.
Here is one example. The first apply version runs 19 seconds, the
second loop version runs 13 seconds, the third matrix version runs 1
second.
Jason Liao
rm(list=ls(all=TRUE))
test1 <- function()
{
u <- runif(n*m);
dim(u) <- c(n,m);
v <- apply(u, 2, crossprod);
}
test2 <- function()
{
u <- runif(n*m);
dim(u) <- c(n,m);
v <- numeric(m);
for(j in 1:m) v[j] <- crossprod(u[, j]);
}
test3 <- function()
{
u <- runif(n*m);
dim(u) <- c(n,m);
u <- u*u;
one <- rep(1, n);
v <- one %*% u;
}
n<-10;
m<-10000;
print(date());
for(i in 1:10) test1()
print(date());
for(i in 1:10) test2()
print(date());
for(i in 1:10) test3()
print(date());
====Jason G. Liao
Department of Biometry and Epidemiology
Medical University of South Carolina
135 Rutledge Ave., STE 1148, Charleston, SC 29425
phone (843) 876-1114, fax (843) 876-1126
http://www.geocities.com/jg_liao/index.html
__________________________________________________
Do You Yahoo!?
Yahoo! Auctions - buy the things you want at great prices
http://auctions.yahoo.com/
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !) To: r-help-request at
stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Jason,
Instead of using print(date()) to time your tests, you can use
system.time(). If you want the time of the whole process, it is the third
element in the vector returned form system.time(). Thus, you could write:
system.time(for(i in 1:10) {test1()})[3]
It is well-known that matrix calculation is much faster than loops in a
matrix calculation package (!) and R is one. I am surprised that apply() is
slower than for loops. Unfortunatelly, matrix algorithm does not always
exist. An alternative being to use C or fortran code which are much faster
in loops. I personnally use another method which does not require so much
change in the code and keep the advantage of matrix calculation where
required, while speeding up loops by at least a factor 10: I transfer
calculation to Ox, a very fast, light-weight matrix calculation package. You
should find a reference to this package on the web.
Best regards,
Philippe Grosjean
-----Message d'origine-----
De : owner-r-help at stat.math.ethz.ch
[mailto:owner-r-help at stat.math.ethz.ch]De la part de Jason Liao
Envoye : mardi 1 mai 2001 17:32
A : r-help at lists.R-project.org
Objet : [R] 6 times faster by eliminating apply
This is some kind of follow-up to my previous posts. I have further
improved the speed of my program 6 times by eliminating all the
apply(). It turns out that apply is slow, is slower than direct loop,
it is an order slower than a matrix operation alternative.
Here is one example. The first apply version runs 19 seconds, the
second loop version runs 13 seconds, the third matrix version runs 1
second.
Jason Liao
rm(list=ls(all=TRUE))
test1 <- function()
{
u <- runif(n*m);
dim(u) <- c(n,m);
v <- apply(u, 2, crossprod);
}
test2 <- function()
{
u <- runif(n*m);
dim(u) <- c(n,m);
v <- numeric(m);
for(j in 1:m) v[j] <- crossprod(u[, j]);
}
test3 <- function()
{
u <- runif(n*m);
dim(u) <- c(n,m);
u <- u*u;
one <- rep(1, n);
v <- one %*% u;
}
n<-10;
m<-10000;
print(date());
for(i in 1:10) test1()
print(date());
for(i in 1:10) test2()
print(date());
for(i in 1:10) test3()
print(date());
====Jason G. Liao
Department of Biometry and Epidemiology
Medical University of South Carolina
135 Rutledge Ave., STE 1148, Charleston, SC 29425
phone (843) 876-1114, fax (843) 876-1126
http://www.geocities.com/jg_liao/index.html
__________________________________________________
Do You Yahoo!?
Yahoo! Auctions - buy the things you want at great prices
http://auctions.yahoo.com/
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.
-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !) To: r-help-request at
stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._.
_._
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !) To: r-help-request at
stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Philippe Grosjean wrote (in reply to Jason Liao):>It is well-known that matrix calculation is much faster than loops in a >matrix calculation package (!) and R is one. I am surprised that apply() is >slower than for loops. Unfortunatelly, matrix algorithm does not always >exist. An alternative being to use C or fortran code which are much faster >in loops. I personnally use another method which does not require so much >change in the code and keep the advantage of matrix calculation where >required, while speeding up loops by at least a factor 10: I transfer >calculation to Ox, a very fast, light-weight matrix calculation package. You >should find a reference to this package on the web.Is there a good way to organize co-operation between R and Ox? (incidentally, Ox is at http://www.nuff.ox.ac.uk/Users/Doornik/) -- David Firth Phone +44 1865 278544 Nuffield College Fax +44 1865 278621 Oxford OX1 1NF Secretary +44 1865 278553 United Kingdom Email david.firth at nuffield.ox.ac.uk http://www.stats.ox.ac.uk/~firth/ -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._