thr3ads.net - R help - [R] Aggregate weights for a unique set of rows [Jun 2012]

If this information is useful, please help other people find it:
Share via:

Weiser, Constantin

2012-Jun-28 13:06 UTC

[R] Aggregate weights for a unique set of rows

Hi, all together. I have - a maybe trivial - problem with aggregating a
list of weights.

Here is the problem:
 - At first I have set of nodes (X/Y-coordinates) and associated weights,
where the set
   of nodes is typically not unique
 - I want to get a set of unique nodes and the sum of associated weights

I am grateful for any help


See for example:

	# weights:
	w <- c(1, 1, 1, 1, 1)

	# not unique set of nodes (X/Y-coordinates):
	nodes <- matrix(c(1,2,3,4,5,6,1,2,1,4),ncol=2, byrow=T)


desired Result:

	#nodes
	    [,1] [,2]
	[1,]    1    2
	[2,]    3    4
	[3,]    5    6
	[4,]    1    4



	#weights
	2 1 1 1



That is my solution, but it is very slow (typical size of nodes -->
200000x2):

weights <- c(1, 1, 1, 1, 1)
nodes <- matrix(c(1,2,3,4,5,6,1,2,1,4),ncol=2, byrow=T)


## to be replaced by a faster code
	drop.index <- duplicated(nodes)
	n.unique <- nodes[!drop.index, ]
	w.unique <- numeric(length(n.unique[,1]))

	lw <- length(weights)
	for (i in seq_along(w.unique)){
		index <- as.logical(2==rowSums(nodes==matrix(rep(n.unique[i,],lw),byrow
= TRUE, nrow=lw)))
		w.unique[i] <- sum(weights[index])
	}
##

n.unique
w.unique





^
|                X
|               /
|              /eiser, Constantin
|             / Gutenberg University of Mainz, Germany
| *    /\    /  Chair of Statistics & Econometrics
|  \  /  \  /   Jakob-Welder-Weg 4, 55128 Mainz
|   \/    \/    House of Law and Economics II, Room 00-116
|               Tel: 0049 6131 39 22715
+--------------------------------------------------------->

Jean V Adams

2012-Jun-28 19:19 UTC

head link

[R] Aggregate weights for a unique set of rows

You say "typical size of nodes --> 200000x2"
Do you mean that nodes has many many rows?  Or many many columns?  Or 
both?

This is minimalist coding, but I'm not sure how fast it will run on your 
data
        aggregate(w, as.data.frame(nodes), sum)

Jean


"Weiser, Constantin" <constantin.weiser@uni-mainz.de> wrote on
06/28/2012
08:06:27 AM:
> Hi, all together. I have - a maybe trivial - problem with aggregating a
> list of weights.
> 
> Here is the problem:
>  - At first I have set of nodes (X/Y-coordinates) and associated 
weights,> where the set
>    of nodes is typically not unique
>  - I want to get a set of unique nodes and the sum of associated weights
> 
> I am grateful for any help
> 
> 
> See for example:
> 
>    # weights:
>    w <- c(1, 1, 1, 1, 1)
> 
>    # not unique set of nodes (X/Y-coordinates):
>    nodes <- matrix(c(1,2,3,4,5,6,1,2,1,4),ncol=2, byrow=T)
> 
> 
> desired Result:
> 
>    #nodes
>        [,1] [,2]
>    [1,]    1    2
>    [2,]    3    4
>    [3,]    5    6
>    [4,]    1    4
> 
> 
> 
>    #weights
>    2 1 1 1
> 
> 
> 
> That is my solution, but it is very slow (typical size of nodes -->
> 200000x2):
> 
> weights <- c(1, 1, 1, 1, 1)
> nodes <- matrix(c(1,2,3,4,5,6,1,2,1,4),ncol=2, byrow=T)
> 
> 
> ## to be replaced by a faster code
>    drop.index <- duplicated(nodes)
>    n.unique <- nodes[!drop.index, ]
>    w.unique <- numeric(length(n.unique[,1]))
> 
>    lw <- length(weights)
>    for (i in seq_along(w.unique)){
>       index <- 
as.logical(2==rowSums(nodes==matrix(rep(n.unique[i,],lw),byrow> = TRUE, nrow=lw)))
>       w.unique[i] <- sum(weights[index])
>    }
> ##
> 
> n.unique
> w.unique
> 
> 
> 
> 
> 
> ^
> |                X
> |               /
> |              /eiser, Constantin
> |             / Gutenberg University of Mainz, Germany
> | *    /\    /  Chair of Statistics & Econometrics
> |  \  /  \  /   Jakob-Welder-Weg 4, 55128 Mainz
> |   \/    \/    House of Law and Economics II, Room 00-116
> |               Tel: 0049 6131 39 22715
> +--------------------------------------------------------->
	[[alternative HTML version deleted]]

Rui Barradas

2012-Jun-29 00:04 UTC

head link

[R] Aggregate weights for a unique set of rows

Hello,

Would around two orders of magnitude interess?

f1 <- function(Nodes, Weights){
	drop.index <- duplicated(Nodes)
	n.unique <- Nodes[!drop.index, ]
	w.unique <- numeric(length(n.unique[,1]))

	lw <- length(Weights)
	for (i in seq_along(w.unique)){
		index <- as.logical(2 == rowSums(
			Nodes == matrix(rep(n.unique[i,],lw), byrow=TRUE, nrow=lw)))
		w.unique[i] <- sum(Weights[index])
	}
	list(n.unique=n.unique, w.unique=w.unique)
}

f2 <- function(Nodes, Weights){
	rows <- paste(Nodes[,1], Nodes[,2], sep=".")
	w.uniq <- tapply(Weights, rows, sum)
	attributes(w.uniq) <- NULL
	ord <- order(unique(rows))
	list(n.unique=unique(Nodes), w.unique=w.uniq[order(ord)])
}

# Test it
M <- 100  # see text below
n <- 2e5

set.seed(1234)
nd <- matrix(sample(M, n*2, TRUE), n, 2)
ww <- rep(1, n)

t1 <- system.time(r1 <- f1(nd, ww))
t2 <- system.time(r2 <- f2(nd, ww))
identical(r1, r2)

print(rbind(t1=t1, t2=t2, ratio=t1/t2), digits=3)
       user.self sys.self elapsed user.child sys.child
t1       310.41       67  379.07         NA        NA
t2         5.59        0    5.62         NA        NA
ratio     55.53      Inf   67.45         NA        NA


With bigger M the number of pairwise combinations increases and so does 
the number of unique rows. This causes the time taken by f1 to really 
increase, but f2 scales up rather slowly. The ratio above becomes really 
better and better.

Hope this helps,

Rui Barradas


Em 28-06-2012 14:06, Weiser, Constantin escreveu:> Hi, all together. I have - a maybe trivial - problem with aggregating a
> list of weights.
>
> Here is the problem:
>   - At first I have set of nodes (X/Y-coordinates) and associated weights,
> where the set
>     of nodes is typically not unique
>   - I want to get a set of unique nodes and the sum of associated weights
>
> I am grateful for any help
>
>
> See for example:
>
> 	# weights:
> 	w <- c(1, 1, 1, 1, 1)
>
> 	# not unique set of nodes (X/Y-coordinates):
> 	nodes <- matrix(c(1,2,3,4,5,6,1,2,1,4),ncol=2, byrow=T)
>
>
> desired Result:
>
> 	#nodes
> 	    [,1] [,2]
> 	[1,]    1    2
> 	[2,]    3    4
> 	[3,]    5    6
> 	[4,]    1    4
>
>
>
> 	#weights
> 	2 1 1 1
>
>
>
> That is my solution, but it is very slow (typical size of nodes -->
> 200000x2):
>
> weights <- c(1, 1, 1, 1, 1)
> nodes <- matrix(c(1,2,3,4,5,6,1,2,1,4),ncol=2, byrow=T)
>
>
> ## to be replaced by a faster code
> 	drop.index <- duplicated(nodes)
> 	n.unique <- nodes[!drop.index, ]
> 	w.unique <- numeric(length(n.unique[,1]))
>
> 	lw <- length(weights)
> 	for (i in seq_along(w.unique)){
> 		index <-
as.logical(2==rowSums(nodes==matrix(rep(n.unique[i,],lw),byrow
> = TRUE, nrow=lw)))
> 		w.unique[i] <- sum(weights[index])
> 	}
> ##
>
> n.unique
> w.unique
>
>
>
>
>
> ^
> |                X
> |               /
> |              /eiser, Constantin
> |             / Gutenberg University of Mainz, Germany
> | *    /\    /  Chair of Statistics & Econometrics
> |  \  /  \  /   Jakob-Welder-Weg 4, 55128 Mainz
> |   \/    \/    House of Law and Economics II, Room 00-116
> |               Tel: 0049 6131 39 22715
> +--------------------------------------------------------->
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

Maybe Matching Threads

Search for more possibly parallel threads

R help - Jun 2012 - Aggregate weights for a unique set of rows

[R] Aggregate weights for a unique set of rows

[R] Aggregate weights for a unique set of rows

[R] Aggregate weights for a unique set of rows

Maybe Matching Threads