Anyone got any hints on how to make this code more efficient? An early
version (which to be fair did more than this one is) ran for 330 hours
and produced no output.
I have a two column table, Dat, with 12,000,000 rows and I want to
produce a lookup table, ltable, in a 1 dimensional matrix with one
copy of each of the values in Dat:
for (i in 1:nrow(Dat))
{
for (j in 1:2)
{
#If next value is already in ltable, do nothing
if (is.na(match(Dat[i,j], ltable))){ltable <- rbind(ltable,Dat[i,j])}
}
}
but it takes forever to produce anything.
Any advice gratefully received.
Thomas
?unique x <- matrix(c(1:6, 6:1),ncol=2) x.temp <- x dim(x.temp) <- NULL unique(x.temp) Michael On Thu, Oct 6, 2011 at 8:37 AM, Thomas <chesney.alt at gmail.com> wrote:> Anyone got any hints on how to make this code more efficient? An early > version (which to be fair did more than this one is) ran for 330 hours and > produced no output. > > I have a two column table, Dat, with 12,000,000 rows and I want to produce a > lookup table, ltable, in a 1 dimensional matrix with one copy of each of the > values in Dat: > > for (i in 1:nrow(Dat)) > { > for (j in 1:2) > { > #If next value is already in ltable, do nothing > if (is.na(match(Dat[i,j], ltable))){ltable <- rbind(ltable,Dat[i,j])} > } > } > > but it takes forever to produce anything. > > Any advice gratefully received. > > Thomas > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >
Probably most of the time you're waiting for this you are in Circle 2 of 'The R Inferno'. If the values are numbers, you might also be in Circle 1. On 06/10/2011 13:37, Thomas wrote:> Anyone got any hints on how to make this code more efficient? An early > version (which to be fair did more than this one is) ran for 330 hours > and produced no output. > > I have a two column table, Dat, with 12,000,000 rows and I want to > produce a lookup table, ltable, in a 1 dimensional matrix with one copy > of each of the values in Dat: > > for (i in 1:nrow(Dat)) > { > for (j in 1:2) > { > #If next value is already in ltable, do nothing > if (is.na(match(Dat[i,j], ltable))){ltable <- rbind(ltable,Dat[i,j])} > } > } > > but it takes forever to produce anything. > > Any advice gratefully received. > > Thomas > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >-- Patrick Burns pburns at pburns.seanet.com twitter: @portfolioprobe http://www.portfolioprobe.com/blog http://www.burns-stat.com (home of 'Some hints for the R beginner' and 'The R Inferno')
Making a bit more sense now: "If you are translating code into R that has a double for loop, think." The R Inferno, Page 18. -- View this message in context: http://r.789695.n4.nabble.com/Running-slow-tp3878093p3881951.html Sent from the R help mailing list archive at Nabble.com.