R help - Apr 2020 - repeat rows of matrix by number (k) in one colummatrix adding column j with values 1:k

Hi Nevil,
It's a nasty piece of work, but:

M<-matrix(c(1,2,3,4,1,2,3,4,1,2,3,4,2,1,3,2),4,4,
dimnames = list(NULL, c("x", "y",
"z","k")))
M
reprow<-function(x)
 return(matrix(rep(x,x[length(x)]),nrow=x[length(x)],byrow=TRUE))
toseq<-function(x) return(1:x)
j<-unlist(sapply(M[,"k"],toseq))
Mlist<-apply(M,1,reprow)
Mlist
newM<-Mlist[[1]]
for(i in 2:length(Mlist)) newM<-rbind(newM,Mlist[[i]])
newM
newM<-cbind(newM,j)
colnames(newM)<-c(colnames(M),"j")
newM

Jim

On Wed, Apr 1, 2020 at 1:43 PM nevil amos <nevil.amos at gmail.com>
wrote:
>
> Well,
> I found a way to do it partly using rep(), and one loop that makes it 10x
> or more faster however would still be good to do without the loop at all
> matrix made slightly beigger (10000 rows):
>
> M<-matrix(c(1:30000
> ), 10000,3)
> M<-cbind(M,sample(1:5,size = 10000,replace = T))
> #Print(M)
> #Create matrix (Mout) in this case 8 rows with x,y,z in each row of M
> #repeated k times with column j numbered from 1:k
> # ! can do with nested loops but this is very slow ( example below)
> #How do I acheive this quickly without loops?
> print("double loop")
> print(system.time({
> Mout<-NULL
>
> for(i in 1:nrow(M)){
>   a=M[i,1:3]
>   k=M[i,4]
>   for (j in 1:k){
>     b=c(a,j)
>     Mout<-rbind(Mout,b)
>     }
> }
>
colnames(Mout)<-c("x","y","z","j")
> }))
>
> print("rep and single loop")
> print(system.time({
>   j<-NULL
>   MOut<-NULL
>   MOut<-M[,1:3][rep(1:nrow(M), times = M[,4]), ]
>   for(i in M[,4])(j<-c(j,1:i))
>   MOut<-cbind(MOut,j)
>  
colnames(Mout)<-c("x","y","z","j")
> }))
>
> On Wed, 1 Apr 2020 at 12:18, nevil amos <nevil.amos at gmail.com>
wrote:
>
> > Hi
> >
> > I can achieve this using two for loops but it is slow  I need to do
this
> > on many matrices with tens of millions of rows of x,y,z and k
> >
> > What is a faster method to achieve this, I cannot use rep as j changes
in
> > each row of the new matrix
> > ###############################################
> > M<-matrix(c(1,2,3,4,1,2,3,4,1,2,3,4, 2, 1, 3, 2
> > ), 4,4, dimnames = list(NULL, c("x", "y",
"z","k")))
> >
> > Print(M)
> > #Create matrix (Mout) in this case 8 rows with x,y,z in each row of M
> > #repeated k times with column j numbered from 1:k
> > # ! can do with nested loops but this is very slow ( example below)
> > #How do I acheive this quickly without loops?
> > Mout<-NULL
> >
> > for(i in 1:nrow(M)){
> >   a=M[i,c("x","y","z")]
> >   for (j in 1:M[i,"k"]){
> >     b=c(a,j)
> >     Mout<-rbind(Mout,b)
> >     }
> > }
> > colnames(Mout)[4]<-"j"
> > print(Mout)
> >
> > #########################################################
> >
> > Thanks
> >
> > Nevil Amos
> >
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

On 2020-04-01 15:33 +1100, Jim Lemon wrote:
> Hi Nevil,
> It's a nasty piece of work, but:
Hi!  How about this one:

    data <- c(rep(1:4, times=3), 2, 1, 3, 2)
    dimnames <- list(NULL, c("x", "y", "z",
"k"))
    ncol <- length(data)/4
    M <- matrix(data=data, nrow=ncol, ncol=ncol, dimnames=dimnames)
    
    FUN <- function(x) {
      out <- matrix(rep(x[1:3], times=x[4]),
                    byrow=TRUE, nrow=x[4])
      return(cbind(out, x[4]))
    }
    newM <- do.call(rbind, apply(X=M, MARGIN=1, FUN=FUN))
    dimnames(newM) <- list(NULL, c("x", "y",
"z", "k"))
    newM <- cbind(newM, "j"=sequence(M[,"k"]))
    newM

Regards,
Rasmus

Hi Rasmus,
Very nice. The R help list is a bit like a quiz show where the
contestants must balance the alacrity with which they press their
buttons with the confidence that they have the correct answer. A slow
motion game of wits in which the prizes are self-awarded.

Jim

On Fri, Apr 3, 2020 at 7:04 AM Rasmus Liland <jensrasmus at gmail.com>
wrote:
>
> On 2020-04-01 15:33 +1100, Jim Lemon wrote:
> > Hi Nevil,
> > It's a nasty piece of work, but:
>
> Hi!  How about this one:
>
>     data <- c(rep(1:4, times=3), 2, 1, 3, 2)
>     dimnames <- list(NULL, c("x", "y",
"z", "k"))
>     ncol <- length(data)/4
>     M <- matrix(data=data, nrow=ncol, ncol=ncol, dimnames=dimnames)
>
>     FUN <- function(x) {
>       out <- matrix(rep(x[1:3], times=x[4]),
>                     byrow=TRUE, nrow=x[4])
>       return(cbind(out, x[4]))
>     }
>     newM <- do.call(rbind, apply(X=M, MARGIN=1, FUN=FUN))
>     dimnames(newM) <- list(NULL, c("x", "y",
"z", "k"))
>     newM <- cbind(newM, "j"=sequence(M[,"k"]))
>     newM
>
> Regards,
> Rasmus