HI Bert and Ben,
Yes, running lm.fit using the matrix format is much faster. I read a couple of
online comments why it is faster.
However, the residual values for three tested variables or genes from lm
function and lm.fit function are different, with Pearson correlation of 0.55,
0.89, and 0.99.
I have not found the reason.
Thanks,
Ding
From: Bert Gunter <bgunter.4567 at gmail.com>
Sent: Friday, August 9, 2024 7:11 PM
To: Ben Bolker <bbolker at gmail.com>
Cc: Yuan Chun Ding <ycding at coh.org>; r-help at r-project.org
Subject: Re: [R] a fast way to do my job
Better idea, Ben! It would work as you might expect it to to produce the same
results as the above: ##first make sure your regressor is a matrix: pur2 <-
matrix(purity2, ncol =1) ## convert the data frame variables into a matrix dat
<-
Better idea, Ben!
It would work as you might expect it to to produce the same results as
the above:
##first make sure your regressor is a matrix:
pur2 <- matrix(purity2, ncol =1)
## convert the data frame variables into a matrix
dat <- as.matrix(gem751be.rpkm[ , 74:35164])
##then
result <- residuals(lm.fit( x= pur2, y = dat))
Cheers,
Bert
On Fri, Aug 9, 2024 at 6:38?PM Ben Bolker <bbolker at
gmail.com<mailto:bbolker at gmail.com>> wrote:
>
> You can also fit a linear model with a matrix-valued response
> variable, which should be even faster (not sure off the top of my head
> how to get the residuals and reshape them to the dimensions you want)
>
> On Fri, Aug 9, 2024 at 9:31?PM Bert Gunter <bgunter.4567 at
gmail.com<mailto:bgunter.4567 at gmail.com>> wrote:
> >
> > See ?lm.fit.
> > I must be missing something, because:
> >
> > results <- sapply(74:35164, \(i) residuals(lm.fit(purity2,
> > gem751be.rpkm[, i] )))
> >
> > would give you a 751 x 35091 matrix of the residuals from each of the
> > regressions.
> > I assume it will be considerably faster than all the overhead you are
> > carrying in your current code, but of course you'll have to try it
and
> > see. ... Assuming that I have interpreted your request correctly.
> > Ignore if not.
> >
> > Cheers,
> > Bert
> >
> > On Fri, Aug 9, 2024 at 4:50?PM Yuan Chun Ding via R-help
> > <r-help at r-project.org<mailto:r-help at r-project.org>>
wrote:
> > >
> > > Dear R users,
> > >
> > > I am running the following code below, the gem751be.rpkm is a
dataframe with dim of 751 samples by 35164 variables, 73 phenotypic variables
in the furst to 73rd column and 35091 genomic variables or genes in the 74th to
35164th columns. What I need to do is to calculate the residuals for each gene
using the simple linear regression model of genelist[i] ~ purity2;
> > >
> > > The following code is running, it takes long time, but I have an
expensive ThinkStation window computer.
> > > Can you provide a fast way to do it?
> > >
> > > Thank you,
> > >
> > > Ding
> > >
> > >
---------------------------------------------------------------------------------
> > >
> > >
> > > gem751be.rpkm <-merge(gem751be10,
as.data.frame(t(rna849.fpkm2)),
> > > + by.x="id2",by.y=0)
> > > > row.names(gem751be.rpkm)<-gem751be.rpkm$id3
> > > >
colnames(gem751be.rpkm)<-gsub(colnames(gem751be.rpkm),pattern="-",replacement="_")
> > > > genelist <- gem751be.rpkm %>%
dplyr::select(74:35164)
> > > > residuals <- NULL
> > > > for (i in 1:length(genelist)) {
> > > + #i=1
> > > + formula <- reformulate("purity2",
response=names(genelist)[i])
> > > + model <- lm(formula, data = gem751be.rpkm)
> > > + resi <- as.data.frame(residuals(model))
> > > + colnames(resi)[1]<-names(genelist)[i]
> > > + resi <-as.data.frame(t(resi))
> > > + residuals <- rbind(residuals, resi)
> > > + }
> > >
> > >
> > >
> > >
----------------------------------------------------------------------
> > > ------------------------------------------------------------
> > > -SECURITY/CONFIDENTIALITY WARNING-
> > >
> > > This message and any attachments are intended solely for the
individual or entity to which they are addressed. This communication may contain
information that is privileged, confidential, or exempt from disclosure under
applicable law (e.g., personal health information, research data, financial
information). Because this e-mail has been sent without encryption, individuals
other than the intended recipient may be able to view the information, forward
it to others or tamper with the information without the knowledge or consent of
the sender. If you are not the intended recipient, or the employee or person
responsible for delivering the message to the intended recipient, any
dissemination, distribution or copying of the communication is strictly
prohibited. If you received the communication in error, please notify the sender
immediately by replying to this message and deleting the message and any
accompanying files from your system. If, due to the security risks, you do not
wish to rec
> > > eive further communications via e-mail, please reply to this
message and inform the sender that you do not wish to receive further e-mail
from the sender. (LCP301)
> > > ------------------------------------------------------------
> > >
> > > [[alternative HTML version deleted]]
> > >
> > > ______________________________________________
> > > R-help at r-project.org<mailto:R-help at r-project.org>
mailing list -- To UNSUBSCRIBE and more, see
> > >
https://urldefense.com/v3/__https://stat.ethz.ch/mailman/listinfo/r-help__;!!Fou38LsQmgU!qcJ4z-vlMNzsa8XCsJUcuPOz8Vt12zsV_XaWpqXsyUYJBTlcNRonFPr7w7Ql3xqcDnZ9ZYC8JX72PW30DQ$<https://urldefense.com/v3/__https:/stat.ethz.ch/mailman/listinfo/r-help__;!!Fou38LsQmgU!qcJ4z-vlMNzsa8XCsJUcuPOz8Vt12zsV_XaWpqXsyUYJBTlcNRonFPr7w7Ql3xqcDnZ9ZYC8JX72PW30DQ$>
> > > PLEASE do read the posting guide
https://urldefense.com/v3/__http://www.R-project.org/posting-guide.html__;!!Fou38LsQmgU!qcJ4z-vlMNzsa8XCsJUcuPOz8Vt12zsV_XaWpqXsyUYJBTlcNRonFPr7w7Ql3xqcDnZ9ZYC8JX66rfmKvA$<https://urldefense.com/v3/__http:/www.R-project.org/posting-guide.html__;!!Fou38LsQmgU!qcJ4z-vlMNzsa8XCsJUcuPOz8Vt12zsV_XaWpqXsyUYJBTlcNRonFPr7w7Ql3xqcDnZ9ZYC8JX66rfmKvA$>
> > > and provide commented, minimal, self-contained, reproducible
code.
> >
> > ______________________________________________
> > R-help at r-project.org<mailto:R-help at r-project.org> mailing
list -- To UNSUBSCRIBE and more, see
> >
https://urldefense.com/v3/__https://stat.ethz.ch/mailman/listinfo/r-help__;!!Fou38LsQmgU!qcJ4z-vlMNzsa8XCsJUcuPOz8Vt12zsV_XaWpqXsyUYJBTlcNRonFPr7w7Ql3xqcDnZ9ZYC8JX72PW30DQ$<https://urldefense.com/v3/__https:/stat.ethz.ch/mailman/listinfo/r-help__;!!Fou38LsQmgU!qcJ4z-vlMNzsa8XCsJUcuPOz8Vt12zsV_XaWpqXsyUYJBTlcNRonFPr7w7Ql3xqcDnZ9ZYC8JX72PW30DQ$>
> > PLEASE do read the posting guide
https://urldefense.com/v3/__http://www.R-project.org/posting-guide.html__;!!Fou38LsQmgU!qcJ4z-vlMNzsa8XCsJUcuPOz8Vt12zsV_XaWpqXsyUYJBTlcNRonFPr7w7Ql3xqcDnZ9ZYC8JX66rfmKvA$<https://urldefense.com/v3/__http:/www.R-project.org/posting-guide.html__;!!Fou38LsQmgU!qcJ4z-vlMNzsa8XCsJUcuPOz8Vt12zsV_XaWpqXsyUYJBTlcNRonFPr7w7Ql3xqcDnZ9ZYC8JX66rfmKvA$>
> > and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]