>>>>> "UweL" == Uwe Ligges <ligges at
statistik.uni-dortmund.de>
>>>>> on Mon, 26 Nov 2007 22:14:07 +0100 writes:
UweL> tomas.larsson at gm.com wrote:
>> Full_Name: Tomas Larsson
>> Version: 2.6.0
>> OS: Windows XP
>> Submission from: (NULL) (198.208.251.24)
>>
>>
>> This is not a bug, it is a performance issue but I think it should
have an easy
>> fix.
>>
>> I have a large matrix (about 2,000,000 by 20), when I type
colnames(x) it takes
>> a long time to get the result. However, if I select just the first
couple of
>> rows of the matrix I don't have to wait for the result. See
below for example.
>>> system.time(colnames(x))
>> user system elapsed
>> 9.98 0.00 10.00
>>> system.time(colnames(x[1:2,]))
>> user system elapsed
>> 0.01 0.00 0.02
UweL> Documentation in the released version of R (2.6.1) tells us:
UweL> For a data frame, 'rownames'
UweL> and 'colnames' are calls to 'row.names' and
'names' respectively,
UweL> but the latter are preferred.
aaah, so we do have a something close to a bug,
since the above is only correct if "are" is interpreted in quite
a wide sense :
Both colnames() and rownames() call dimnames()
which is a (.Primitive) generic and the data.frame method simply
is
function (x) list(row.names(x), names(x))
So, in fact, both colnames() and rownames()
each call *both* row.names() and names() even though only one of
them is needed.
This is indeed suboptimal in the case where colnames() is of
length 20 and rownames() is of length 2'000'000 ... not really
an atypical case.
And that (last paragraph) is also true when 'x' is a matrix and
not a data.frame. However, there's a bit more to it...
UweL> and on my machine I get:
UweL> system.time(names(x))
UweL> user system elapsed
UweL> 0 0 0
yes. But what if his 'x' really *was* a matrix (and not a data frame)?
The speed of colnames(x) in such a case depends quite a bit if the
matrix has non-NULL rownames. Ideally I think it should not,
and hence partly agree with Tomas.
HOWEVER, there's more to it.
If 'x' *was* a matrix --- and this proves to me that it was not
in Tomas' case ---
even though colnames() seems like a waste (of memory, copying),
it is infact still very fast in newer versions of R ... most probably
because 'character' vectors are hashed now and much less memory
allocation is happening than in earlier versions of R.
The only case that is slow is for a *data frame* with
"empty" i.e. automatic rownames. Watch this :
m <- matrix(pi, 2e6, 20,
dimnames=list(LETTERS[sample(26,2e6,replace=TRUE)], letters[1:20]))
system.time(for(i in 1:100) cc <- colnames(m))
## 0.001 -- very fast
## ditto for this:
system.time(for(i in 1:100) dd <- dimnames(m))
## HOWEVER:
system.time(dm <- as.data.frame(m)) ## takes more than a second
## user system elapsed
## 2.462 1.379 3.842
## Quite a bit slower (x 1000 !) than for the matrix above, but still ok:
system.time(for(i in 1:100) c2 <- colnames(dm))
## user system elapsed
## 1.202 0.638 1.842
stopifnot(identical(c2, cc))
## ditto
system.time(for(i in 1:100) d2 <- dimnames(dm))
## user system elapsed
## 1.143 0.626 1.769
stopifnot(identical(d2, dd))
###---- BUT now: What happens if we have "empty" rownames ???
## m0 := {m with empty rownames} :
m0 <- m
dimnames(m0) <- list(NULL, colnames(m0))
## and ditto for the data frames:
## dm0 := {dm with empty rownames, i.e. "internal/automatic 1:N
rownames}:
system.time(dm0 <- as.data.frame(m0))
## user system elapsed
## 1.677 1.241 2.922
system.time(c3 <- colnames(dm0))
## user system elapsed
## 5.208 0.047 5.261
###---> OOOPS! One single call to colnames(.)
### needs more than 100 calls in the non-empty rownames case
## repeated calls become faster ..... and ....
system.time(c3 <- colnames(dm0))
## user system elapsed
## 3.109 0.000 3.110
## ..... faster ......and even much faster
system.time(c3 <- colnames(dm0))
## user system elapsed
## 0.913 0.007 0.922
## Note: repeated calls to dimnames(.) here become faster :
system.time(d3 <- dimnames(dm0))
## Note indeed, that names() is lightning fast in comparison:
system.time(for(i in 1:100) c4 <- names(dm0)) ## is 'immediate' (0
sec)
## user system elapsed
## 0.001 0.000 0.000 --- 100 x ~1000 times faster
----------------------------------------------
All things considered, I'd currently propose to add
if(is.data.frame(x) && do.NULL)
return(names(x))
to the beginning of 'colnames'.
We have such clause already at the beginning of
'colnames<-'
.... all of which would suggest to make these generic, but we
have been there before and consciously decided against doing so,
on the ground that 'dimnames' is already generic and
colnames(.) and rownames(.) should really be equivalent to
dimnames(.)[[j]] for j=1 or 2, respectively.
Martin Maechler, ETH Zurich