here is a script I use to make overlapping cut of my input "col.dat"
file into a matrix:
#----------------------------------
col2mat<-function(x, sampsz=220, qsamp=2000) {
m<-matrix(nr=qsamp, nc=sampsz)
for(s in 1:qsamp){
print(s)
for(i in 1:sampsz){
m[s,i] <- x[s+i,1]
}
}
m
}
w<-read.table("col.dat", check.names=FALSE)
m<-col2mat(w)
#----------------------------------
the input file "col.dat" looks like this:
#-----------------------
118
118
122
.
.
.
-90
-84
-120
-108
#------------
contains about 380000 numbers and has a size of 1.5Mb. Not a small
file I understand, but not too large! However I do not use the whole
file at _this_ stage of my task, I use no more than 10% (no more than
150Kb) if it is matter. Well.
Problem: m<-col2mat(w) works toooo slow. One row (that is 220 numbers)
of such a matrix is being formed in ~5 sec! (5*2000 = 10000 sec). it
is slowed in thousands times comparing to analogical C-code, too much
I think.
Q1: is there a better way to create my matrix? (maybe I do some
mistake)
Q2(for developers): if this slowdown is a problem of R's engine, may
we hope the problem will be corrected?
thank you.
Valery A.Khamenya (please reply-to:<news_vkhamenya at chat.ru>)
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !) To: r-help-request at
stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Hi!
Using your original function on some dummy data I get:
(700MHz PIII Linux 2.2.18)
w <- as.matrix(floor(runif(100000,100,999)))
unix.time(m <- time(col2mat(w)))
[1] 22.67 0.69 50.48 0.00 0.00
50 seconds to generate the whole matrix.
If I comment out the print(s) it reduces to:
unix.time(m <- time(col2mat(w)))
[1] 21.03 0.64 21.86 0.00 0.00
But if I change the function definition to
"col2mat" <-
function(x, sampsz=220, qsamp=2000) {
m<-matrix(nr=qsamp, nc=sampsz)
for(s in 1:qsamp){
### print(s)
### for(i in 1:sampsz){
m[s,(1:sampsz)] <- x[s+(1:sampsz),1]
### }
}
m
}
which I consider functional equivalent I can go down to:> unix.time(m <- time(col2mat(w)))
[1] 0.52 0.06 0.59 0.00 0.00
Probably you can get rid of the outer loop in a similar fashion.
Hope that helps,
detlef
On 20-Mar-01 Valery A.Khamenya wrote:> here is a script I use to make overlapping cut of my input
"col.dat"
> file into a matrix:
>
>#----------------------------------
> col2mat<-function(x, sampsz=220, qsamp=2000) {
> m<-matrix(nr=qsamp, nc=sampsz)
> for(s in 1:qsamp){
> print(s)
> for(i in 1:sampsz){
> m[s,i] <- x[s+i,1]
> }
> }
> m
> }
> w<-read.table("col.dat", check.names=FALSE)
> m<-col2mat(w)
>#----------------------------------
>
> the input file "col.dat" looks like this:
>#-----------------------
> 118
> 118
> 122
> .
> .
> .
> -90
> -84
> -120
> -108
>#------------
> contains about 380000 numbers and has a size of 1.5Mb. Not a small
> file I understand, but not too large! However I do not use the whole
> file at _this_ stage of my task, I use no more than 10% (no more than
> 150Kb) if it is matter. Well.
>
> Problem: m<-col2mat(w) works toooo slow. One row (that is 220 numbers)
> of such a matrix is being formed in ~5 sec! (5*2000 = 10000 sec). it
> is slowed in thousands times comparing to analogical C-code, too much
> I think.
>
> Q1: is there a better way to create my matrix? (maybe I do some
> mistake)
>
> Q2(for developers): if this slowdown is a problem of R's engine, may
> we hope the problem will be corrected?
>
> thank you.
>
> Valery A.Khamenya (please reply-to:<news_vkhamenya at chat.ru>)
>
>
>
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.
> -
> r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> Send "info", "help", or "[un]subscribe"
> (in the "body", not the subject !) To: r-help-request at
stat.math.ethz.ch
>
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._.
> _
Detlef Steuer ** Universität der Bw ** 22043 Hamburg
Tel: (0049) (0)40/6541-2819
steuer at unibw-hamburg.de
There is more to life than increasing its speed. - Mahatma Gandhi
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !) To: r-help-request at
stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
"Valery A.Khamenya" wrote:> > here is a script I use to make overlapping cut of my input "col.dat" > file into a matrix: > > #---------------------------------- > col2mat<-function(x, sampsz=220, qsamp=2000) { > m<-matrix(nr=qsamp, nc=sampsz) > for(s in 1:qsamp){ > print(s) > for(i in 1:sampsz){ > m[s,i] <- x[s+i,1] > } > } > m > } > w<-read.table("col.dat", check.names=FALSE) > m<-col2mat(w) > #---------------------------------- > > the input file "col.dat" looks like this: > #----------------------- > 118 > 118 > 122 > . > . > . > -90 > -84 > -120 > -108 > #------------ > contains about 380000 numbers and has a size of 1.5Mb. Not a small > file I understand, but not too large! However I do not use the whole > file at _this_ stage of my task, I use no more than 10% (no more than > 150Kb) if it is matter. Well. > > Problem: m<-col2mat(w) works toooo slow. One row (that is 220 numbers) > of such a matrix is being formed in ~5 sec! (5*2000 = 10000 sec). it > is slowed in thousands times comparing to analogical C-code, too much > I think. > > Q1: is there a better way to create my matrix? (maybe I do some > mistake) > > Q2(for developers): if this slowdown is a problem of R's engine, may > we hope the problem will be corrected? >A1: There is a better way. Vectorizing (and so eliminating the loops) will speed up your function. A2: R works as an interpreter. Loops in interpreted code are slow in most cases. Uwe Ligges -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
"Valery A.Khamenya" schrieb:> > here is a script I use to make overlapping cut of my input "col.dat" > file into a matrix:Possibly the following may help: m <- as.matrix(w) dim(m) <- c(sampsz,qsamp) Thomas Petzoldt -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._