If I have understood the request, I'm not sure that omitting all 0
pairs for each pair of columns makes much sense, but be that as it
may, here's another way to do it by using the 'FUN' argument of
combn
to encapsulate any calculations that you do. I just use cor() as the
calculation -- you can use anything you like that takes two vectors of
0's and 1's and produces fixed length numeric results (or fromm which
you can extract such).
I encapsulated it all in a little function. Note that I first
converted the data frame to a matrix. Because of their generality,
data frames carry a lot of extra baggage that can slow purely numeric
manipulations down.
Anyway, here's the function, 'somecors' (I'm a bad name picker
:( ! )
somecors <- function(dat, func = cor){
dat <- as.matrix(dat)
indx <- seq_len(ncol(dat))
combn(indx, 2, FUN = \(z) {
i <- z[1]; j <- z[2]
k <- dat[, i ] | dat[, j ]
c(z,func(dat[k,i ], dat[k,j ]))
})
}
Results come out as a matrix with combn(ncol(dat),2) columns, the
first 2 rows giving the pair of column numbers for each column,and
then 1 or more rows (possibly extracted) from whatever func you use.
Here's the results for your data formatted to 2 decimal places:
> round(somecors(dat),2)
[,1] [,2] [,3] [,4] [,5] [,6]
[1,] 1.0 1.00 1.00 2.00 2 3.00
[2,] 2.0 3.00 4.00 3.00 4 4.00
[3,] -0.5 -0.41 -0.35 -0.41 NA -0.47
Warning message:
In func(dat[k, i], dat[k, j]) : the standard deviation is zero
The NA and warning comes in the 2,4 pair of columns because after
removing all zero rows in the pair, dat[,4] is all 1's, giving a zero
in the denominator of the cor() calculation -- again, assuming I have
correctly understood your request. If so, this might be something you
need to worry about.
Again, feel free to ignore if I have misinterpreterd or this does not suit.
Cheers,
Bert
On Thu, Jul 25, 2024 at 2:01?PM Rui Barradas <ruipbarradas at sapo.pt>
wrote:>
> ?s 20:47 de 25/07/2024, Yuan Chun Ding escreveu:
> > Hi Rui,
> >
> > You are always very helpful!! Thank you,
> >
> > I just modified your R codes to remove a row with zero values in both
column pair as below for my real data.
> >
> > Ding
> >
> > dat<-gene22mut.coded
> > r <- P <- matrix(NA, nrow = 22L, ncol = 22L,
> > dimnames = list(names(dat), names(dat)))
> >
> > for(i in 1:22) {
> > #i=1
> > x <- dat[[i]]
> > for(j in (1:22)) {
> > #j=2
> > if(i == j) {
> > # there's nothing to test, assign correlation 1
> > r[i, j] <- 1
> > } else {
> > tmp <-cbind(x,dat[[j]])
> > row0 <-rowSums(tmp)
> > tem2 <-tmp[row0!=0,]
> > tmp3 <- cor.test(tem2[,1],tem2[,2])
> > r[i, j] <- tmp3$estimate
> > P[i, j] <- tmp3$p.value
> > }
> > }
> > }
> > r<-as.data.frame(r)
> > P<-as.data.frame(P)
> >
> > From: R-help <r-help-bounces at r-project.org> On Behalf Of Yuan
Chun Ding via R-help
> > Sent: Thursday, July 25, 2024 11:26 AM
> > To: Rui Barradas <ruipbarradas at sapo.pt>; r-help at
r-project.org
> > Subject: Re: [R] please help generate a square correlation matrix
> >
> > HI Rui, Thank you for the help! You did not remove a row if zero
values exist in both column pair, right? Ding From: Rui Barradas
<ruipbarradas@?sapo.?pt> Sent: Thursday, July 25, 2024 11:?15 AM To: Yuan
Chun Ding <ycding@?coh.?org>;
> >
> >
> > HI Rui,
> >
> >
> >
> > Thank you for the help!
> >
> >
> >
> > You did not remove a row if zero values exist in both column pair,
right?
> >
> >
> >
> > Ding
> >
> >
> >
> > From: Rui Barradas <ruipbarradas at sapo.pt<mailto:ruipbarradas
at sapo.pt>>
> >
> > Sent: Thursday, July 25, 2024 11:15 AM
> >
> > To: Yuan Chun Ding <ycding at coh.org<mailto:ycding at
coh.org>>; r-help at r-project.org<mailto:r-help at r-project.org>
> >
> > Subject: Re: [R] please help generate a square correlation matrix
> >
> >
> >
> > ?s 17:?39 de 25/07/2024, Yuan Chun Ding via R-help escreveu: > Hi R
users, > > I generated a square correlation matrix for the dat dataframe
below; > dat<-data.?frame(g1=c(1,0,0,1,1,1,0,0,0), >
g2=c(0,1,0,1,0,1,1,0,0), > g3=c(1,1,0,0,0,1,0,0,0),
> >
> >
> >
> >
> >
> > ?s 17:39 de 25/07/2024, Yuan Chun Ding via R-help escreveu:
> >
> >
> >
> >> Hi R users,
> >
> >
> >
> >>
> >
> >
> >
> >> I generated a square correlation matrix for the dat dataframe
below;
> >
> >
> >
> >> dat<-data.frame(g1=c(1,0,0,1,1,1,0,0,0),
> >
> >
> >
> >> g2=c(0,1,0,1,0,1,1,0,0),
> >
> >
> >
> >> g3=c(1,1,0,0,0,1,0,0,0),
> >
> >
> >
> >> g4=c(0,1,0,1,1,1,1,1,0))
> >
> >
> >
> >> library("Hmisc")
> >
> >
> >
> >> dat.rcorr = rcorr(as.matrix(dat))
> >
> >
> >
> >> dat.r <-round(dat.rcorr$r,2)
> >
> >
> >
> >>
> >
> >
> >
> >> however, I want to modify this correlation calculation;
> >
> >
> >
> >> my dat has more than 1000 rows and 22 columns;
> >
> >
> >
> >> in each column, less than 10% values are 1, most of them are 0;
> >
> >
> >
> >> so I want to remove a row with value of zero in both columns when
calculate correlation between two columns.
> >
> >
> >
> >> I just want to check whether those values of 1 are correlated
between two columns.
> >
> >
> >
> >> Please look at my code in the following;
> >
> >
> >
> >>
> >
> >
> >
> >> cor.4gene <-matrix(0,nrow=4*4, ncol=4)
> >
> >
> >
> >> for (i in 1:4){
> >
> >
> >
> >> #i=1
> >
> >
> >
> >> for (j in 1:4) {
> >
> >
> >
> >> #j=1
> >
> >
> >
> >> d <-dat[,c(i,j)]%>%
> >
> >
> >
> >> filter(eval(as.symbol(colnames(dat)[i]))!=0 |
> >
> >
> >
> >> eval(as.symbol(colnames(dat)[j]))!=0)
> >
> >
> >
> >> c <-cor.test(d[,1],d[,2])
> >
> >
> >
> >> cor.4gene[i*j,]<-c(colnames(dat)[i],colnames(dat)[j],
> >
> >
> >
> >> c$estimate,c$p.value)
> >
> >
> >
> >> }
> >
> >
> >
> >> }
> >
> >
> >
> >> cor.4gene<-as.data.frame(cor.4gene)%>%filter(V1 !=0)
> >
> >
> >
> >>
colnames(cor.4gene)<-c("gene1","gene2","cor","P")
> >
> >
> >
> >>
> >
> >
> >
> >> Can you tell me what mistakes I made?
> >
> >
> >
> >> first, why cor is NA when calculation of correlation for g1 and
g1, I though it should be 1.
> >
> >
> >
> >>
> >
> >
> >
> >> cor.4gene$cor[is.na(cor.4gene$cor)]<-1
> >
> >
> >
> >> cor.4gene$cor[is.na(cor.4gene$P)]<-0
> >
> >
> >
> >> cor.4gene.sq <-pivot_wider(cor.4gene, names_from = gene1,
values_from = cor)
> >
> >
> >
> >>
> >
> >
> >
> >> Then this line of code above did not generate a square matrix as
what the HMisc library did.
> >
> >
> >
> >> How to fix my code?
> >
> >
> >
> >>
> >
> >
> >
> >> Thank you,
> >
> >
> >
> >>
> >
> >
> >
> >> Ding
> >
> >
> >
> >>
> >
> >
> >
> >>
> >
> >
> >
> >>
----------------------------------------------------------------------
> >
> >
> >
> >> ------------------------------------------------------------
> >
> >
> >
> >> -SECURITY/CONFIDENTIALITY WARNING-
> >
> >
> >
> >>
> >
> >
> >
> >> This message and any attachments are intended solely for the
individual or entity to which they are addressed. This communication may contain
information that is privileged, confidential, or exempt from disclosure under
applicable law (e.g., personal health information, research data, financial
information). Because this e-mail has been sent without encryption, individuals
other than the intended recipient may be able to view the information, forward
it to others or tamper with the information without the knowledge or consent of
the sender. If you are not the intended recipient, or the employee or person
responsible for delivering the message to the intended recipient, any
dissemination, distribution or copying of the communication is strictly
prohibited. If you received the communication in error, please notify the sender
immediately by replying to this message and deleting the message and any
accompanying files from your system. If, due to the security risks, you do not
wish to rec
> >
> >
> >
> >> eive further communications via e-mail, please reply to this
message and inform the sender that you do not wish to receive further e-mail
from the sender. (LCP301)
> >
> >
> >
> >> ------------------------------------------------------------
> >
> >
> >
> >>
> >
> >
> >
> >> [[alternative HTML version deleted]]
> >
> >
> >
> >>
> >
> >
> >
> >> ______________________________________________
> >
> >
> >
> >> R-help at r-project.org<mailto:R-help at
r-project.org<mailto:R-help at r-project.org%3cmailto:R-help at
r-project.org>> mailing list -- To UNSUBSCRIBE and more, see
> >
> >
> >
> >>
https://urldefense.com/v3/__https://stat.ethz.ch/mailman/listinfo/r-help__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6Hb8338TBM$<https://urldefense.com/v3/__https:/stat.ethz.ch/mailman/listinfo/r-help__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6Hb8338TBM$><https://urldefense.com/v3/__https:/stat.ethz.ch/mailman/listinfo/r-help__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6Hb8338TBM$%3chttps:/urldefense.com/v3/__https:/stat.ethz.ch/mailman/listinfo/r-help__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6Hb8338TBM$%3e%3e>
> >
> >
<https://urldefense.com/v3/__https:/stat.ethz.ch/mailman/listinfo/r-help__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6Hb8338TBM$%3chttps:/urldefense.com/v3/__https:/stat.ethz.ch/mailman/listinfo/r-help__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6Hb8338TBM$%3e%3e>
> >
> >>
<https://urldefense.com/v3/__https:/stat.ethz.ch/mailman/listinfo/r-help__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6Hb8338TBM$%3chttps:/urldefense.com/v3/__https:/stat.ethz.ch/mailman/listinfo/r-help__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6Hb8338TBM$%3e%3e>PLEASE
do read the posting guide
https://urldefense.com/v3/__http://www.R-project.org/posting-guide.html__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6Hb880tLw0$<https://urldefense.com/v3/__http:/www.R-project.org/posting-guide.html__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6Hb880tLw0$><https://urldefense.com/v3/__http:/www.R-project.org/posting-guide.html__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6Hb880tLw0$%3chttps:/urldefense.com/v3/__http:/www.R-project.org/posting-guide.html__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6Hb880tLw0$%3e%3e>
> >
> >
<https://urldefense.com/v3/__http:/www.R-project.org/posting-guide.html__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6Hb880tLw0$%3chttps:/urldefense.com/v3/__http:/www.R-project.org/posting-guide.html__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6Hb880tLw0$%3e%3e>
> >
> >>
<https://urldefense.com/v3/__http:/www.R-project.org/posting-guide.html__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6Hb880tLw0$%3chttps:/urldefense.com/v3/__http:/www.R-project.org/posting-guide.html__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6Hb880tLw0$%3e%3e>and
provide commented, minimal, self-contained, reproducible code.
> >
> >
> >
> > Hello,
> >
> >
> >
> >
> >
> >
> >
> > You are complicating the code, there's no need for as.symbol/eval,
the
> >
> >
> >
> > column numbers do exactly the same.
> >
> >
> >
> >
> >
> >
> >
> > # create the two results matrices beforehand
> >
> >
> >
> > r <- P <- matrix(NA, nrow = 4L, ncol = 4L, dimnames =
list(names(dat),
> >
> >
> >
> > names(dat)))
> >
> >
> >
> >
> >
> >
> >
> > for(i in 1:4) {
> >
> >
> >
> > x <- dat[[i]]
> >
> >
> >
> > for(j in (1:4)) {
> >
> >
> >
> > if(i == j) {
> >
> >
> >
> > # there's nothing to test, assign correlation 1
> >
> >
> >
> > r[i, j] <- 1
> >
> >
> >
> > } else {
> >
> >
> >
> > tmp <- cor.test(x, dat[[j]])
> >
> >
> >
> > r[i, j] <- tmp$estimate
> >
> >
> >
> > P[i, j] <- tmp$p.value
> >
> >
> >
> > }
> >
> >
> >
> > }
> >
> >
> >
> > }
> >
> >
> >
> >
> >
> >
> >
> > # these two results are equal up to floating-point precision
> >
> >
> >
> > dat.rcorr$r
> >
> >
> >
> > #> g1 g2 g3 g4
> >
> >
> >
> > #> g1 1.0000000 0.1000000 0.3162278 0.1581139
> >
> >
> >
> > #> g2 0.1000000 1.0000000 0.3162278 0.6324555
> >
> >
> >
> > #> g3 0.3162278 0.3162278 1.0000000 0.0000000
> >
> >
> >
> > #> g4 0.1581139 0.6324555 0.0000000 1.0000000
> >
> >
> >
> > r
> >
> >
> >
> > #> g1 g2 g3 g4
> >
> >
> >
> > #> g1 1.0000000 0.1000000 3.162278e-01 1.581139e-01
> >
> >
> >
> > #> g2 0.1000000 1.0000000 3.162278e-01 6.324555e-01
> >
> >
> >
> > #> g3 0.3162278 0.3162278 1.000000e+00 1.355253e-20
> >
> >
> >
> > #> g4 0.1581139 0.6324555 1.355253e-20 1.000000e+00
> >
> >
> >
> >
> >
> >
> >
> > # these two results are equal up to floating-point precision
> >
> >
> >
> > dat.rcorr$P
> >
> >
> >
> > #> g1 g2 g3 g4
> >
> >
> >
> > #> g1 NA 0.79797170 0.4070838 0.68452834
> >
> >
> >
> > #> g2 0.7979717 NA 0.4070838 0.06758329
> >
> >
> >
> > #> g3 0.4070838 0.40708382 NA 1.00000000
> >
> >
> >
> > #> g4 0.6845283 0.06758329 1.0000000 NA
> >
> >
> >
> > P
> >
> >
> >
> > #> g1 g2 g3 g4
> >
> >
> >
> > #> g1 NA 0.79797170 0.4070838 0.68452834
> >
> >
> >
> > #> g2 0.7979717 NA 0.4070838 0.06758329
> >
> >
> >
> > #> g3 0.4070838 0.40708382 NA 1.00000000
> >
> >
> >
> > #> g4 0.6845283 0.06758329 1.0000000 NA
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > You can put these two results in a list, like Hmisc::rcorr does.
> >
> >
> >
> >
> >
> >
> >
> > lst_rcorr <- list(r = r, P = P)
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Hope this helps,
> >
> >
> >
> >
> >
> >
> >
> > Rui Barradas
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > --
> >
> >
> >
> > Este e-mail foi analisado pelo software antiv?rus AVG para verificar a
presen?a de v?rus.
> >
> >
> >
> >
https://urldefense.com/v3/__http://www.avg.com__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6HbloMCQMI$<https://urldefense.com/v3/__http:/www.avg.com__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6HbloMCQMI$><https://urldefense.com/v3/__http:/www.avg.com__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6HbloMCQMI$%3chttps:/urldefense.com/v3/__http:/www.avg.com__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6HbloMCQMI$%3e%09%5b%5balternative>
> >
> >
<https://urldefense.com/v3/__http:/www.avg.com__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6HbloMCQMI$%3chttps:/urldefense.com/v3/__http:/www.avg.com__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6HbloMCQMI$%3e%09%5b%5balternative>
> >
> >
[[alternative<https://urldefense.com/v3/__http:/www.avg.com__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6HbloMCQMI$%3chttps:/urldefense.com/v3/__http:/www.avg.com__;!!Fou38LsQmgU!tyykZkQmOKcwoWXEpV2ohbnr02thhHMabAcYLL_-7dteKHAabK-eo4rGDnwgSFjniAy8SO00L6HbloMCQMI$%3e%09%5b%5balternative>HTML
version deleted]]
> >
> >
> >
> > ______________________________________________
> >
> > R-help at r-project.org<mailto:R-help at r-project.org> mailing
list -- To UNSUBSCRIBE and more, see
> >
> >
https://urldefense.com/v3/__https://stat.ethz.ch/mailman/listinfo/r-help__;!!Fou38LsQmgU!s54ahZtZNAIyaIGV3C2p8lhXpYlHksC6XvFKZltf6g3ElJHOO3I1MYFecLQ4QeMO3MpP3qXz-YrhdUE$<https://urldefense.com/v3/__https:/stat.ethz.ch/mailman/listinfo/r-help__;!!Fou38LsQmgU!s54ahZtZNAIyaIGV3C2p8lhXpYlHksC6XvFKZltf6g3ElJHOO3I1MYFecLQ4QeMO3MpP3qXz-YrhdUE$>
> >
> > PLEASE do read the posting guide
https://urldefense.com/v3/__http://www.R-project.org/posting-guide.html__;!!Fou38LsQmgU!s54ahZtZNAIyaIGV3C2p8lhXpYlHksC6XvFKZltf6g3ElJHOO3I1MYFecLQ4QeMO3MpP3qXzNRZxc6s$<https://urldefense.com/v3/__http:/www.R-project.org/posting-guide.html__;!!Fou38LsQmgU!s54ahZtZNAIyaIGV3C2p8lhXpYlHksC6XvFKZltf6g3ElJHOO3I1MYFecLQ4QeMO3MpP3qXzNRZxc6s$>
> >
> > and provide commented, minimal, self-contained, reproducible code.
> >
> Hello,
>
> Here are two other ways.
>
> The first is equivalent to your long format attempt.
>
>
> library(tidyverse)
>
> dat %>%
> names() %>%
> expand.grid(., .) %>%
> apply(1L, \(x) {
> tmp <- dat[rowSums(dat[x]) > 0, ]
> tmp2 <- cor.test(tmp[[ x[1L] ]], tmp[[ x[2L] ]])
> c(tmp2$estimate, P = tmp2$p.value)
> }) %>%
> t() %>%
> as.data.frame() %>%
> cbind(tmp_df, .) %>%
> na.omit()
>
>
> The second is, in my opinion the one that makes more sense. If you see
> the results, cor is symmetric (as it should) so the calculations are
> repeated. If you only run the cor.tests on the combinations of
> names(dat) by groups of 2, it will save a lot of work. But the output is
> a much smaller a data.frame.
>
>
> cbind(
> combn(names(dat), 2L) %>%
> t() %>%
> as.data.frame(),
> combn(dat, 2L, FUN = \(d) {
> d2 <- d[rowSums(d) > 0, ]
> tmp2 <- cor.test(d2[[1L]], d2[[2L]])
> c(tmp2$estimate, P = tmp2$p.value)
> }) %>% t()
> ) %>% na.omit()
>
>
>
> Hope this helps,
>
> Rui Barradas
>
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.