R help - Apr 2006 - Multivariate linear regression

Hi,

I am working on a multivariate linear regression of the form y = Ax.

I am seeing a great dispersion of y w.r.t x. For example, the
correlations between y and x are very small, even after using some
typical transformations like log, power.

I tried with simple linear regression, robust regression and ace and
avas package in R (or splus). I didn't see an improvement in the fit
and predictions over simple linear regression. (I also tried this with
transformed variables)

I am sure that some of you came across such data. How did you deal with it?

Linear regressions are good for the data like y = x +
0.01Normal(mu,sigma2) i.e. a small noise (data observed in a lab). But
linear regressions are bad for large noise, like typical market (or
survey) data.

Thank you,
Nagu

Ummm... 

If y is unrelated to x, then why would one expect any reasonable method to
show a greater or lesser relationship than any other? It's all random. Of
course, put enough random regressors into/"tune" the parameters enough
of
any regression methodology and you'll be able to precisely predict the data
at hand -- but **only** the data at hand. I should note that such work
apparently frequently appears in various sorts of
"informatics"/"data
mining"/"omics"/etc. journals these days, as various papers
demonstrating
the irreproducibility of numerous purported discoveries have infamously
demonstrated. Let us not forget Occam!

Just being cranky ...

-- Bert Gunter
 
> -----Original Message-----
> From: r-help-bounces at stat.math.ethz.ch 
> [mailto:r-help-bounces at stat.math.ethz.ch] On Behalf Of Nagu
> Sent: Wednesday, April 05, 2006 3:52 PM
> To: r-help at stat.math.ethz.ch
> Subject: [R] Multivariate linear regression
> 
> Hi,
> 
> I am working on a multivariate linear regression of the form y = Ax.
> 
> I am seeing a great dispersion of y w.r.t x. For example, the
> correlations between y and x are very small, even after using some
> typical transformations like log, power.
> 
> I tried with simple linear regression, robust regression and ace and
> avas package in R (or splus). I didn't see an improvement in the fit
> and predictions over simple linear regression. (I also tried this with
> transformed variables)
> 
> I am sure that some of you came across such data. How did you 
> deal with it?
> 
> Linear regressions are good for the data like y = x +
> 0.01Normal(mu,sigma2) i.e. a small noise (data observed in a lab). But
> linear regressions are bad for large noise, like typical market (or
> survey) data.
> 
> Thank you,
> Nagu
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! 
> http://www.R-project.org/posting-guide.html
>

Apparently you do not understand the point, and seem to (want to) see
patterns all over the place. A good start for the treatment of this
interesting disease is 'Fooled by Randomness' by Nassim Nicholas
Taleb. The main point of the book is that many things may be a lot
more random than one might care to imagine or believe. (Ramsey theory
is misleading and of no help here, given its biased premise that
"complete disorder is impossible" (T. S. Motzkin, Wikipedia).)

> -----Original Message-----
> From: r-help-bounces at stat.math.ethz.ch
> [mailto:r-help-bounces at stat.math.ethz.ch] On Behalf Of Nagu
> Sent: Wednesday, April 05, 2006 8:09 PM
> To: Berton Gunter
> Cc: r-help at stat.math.ethz.ch
> Subject: Re: [R] Multivariate linear regression
>
> Hi Bert,
>
> Thank you for your prompt reply.
>
> I understand your point.
>
> But randomness is just a matter of "scale" of the object (Ramsey
> Theory) . The X matrix does not explain the complete variation in Y
> due to a large noise in X or simply the mapping f: X->Y is many valued
> (or due to other finite number of reasons). Theoretically inverse does
> not exist for many valued functions. In regression type problems, we
> are evaluating the pseudoinverse of data space.
>
> To estimate the inverses of many valued functions, theoretically, we
> may have to use branch cuts method or something called Riemann
> surfaces, they are partition of the domain of connected sheets.
>
> As I am not a qualified statistician or have a good experience in
> building statistical models for highly noisy data, I am wondering how
> did you deal with such situations, if any exist, in your working
> experience?
>
> I will try your idea of feeding some random variables as
> predictors in X.
>
> Thank you again,
> Nagu
>
> P.S. Why is that pattern recognition is all about finding patterns
> that can not be seen easily, huh?
>
> On 4/5/06, Berton Gunter <gunter.berton at gene.com> wrote:
> > Ummm...
> >
> > If y is unrelated to x, then why would one expect any
> reasonable method to
> > show a greater or lesser relationship than any other? It's
> all random. Of
> > course, put enough random regressors into/"tune" the
> parameters enough of
> > any regression methodology and you'll be able to precisely
> predict the data
> > at hand -- but **only** the data at hand. I should note
> that such work
> > apparently frequently appears in various sorts of
> "informatics"/"data
> > mining"/"omics"/etc. journals these days, as various
papers
> demonstrating
> > the irreproducibility of numerous purported discoveries
> have infamously
> > demonstrated. Let us not forget Occam!
> >
> > Just being cranky ...
> >
> > -- Bert Gunter
> >
> >
> > > -----Original Message-----
> > > From: r-help-bounces at stat.math.ethz.ch
> > > [mailto:r-help-bounces at stat.math.ethz.ch] On Behalf Of Nagu
> > > Sent: Wednesday, April 05, 2006 3:52 PM
> > > To: r-help at stat.math.ethz.ch
> > > Subject: [R] Multivariate linear regression
> > >
> > > Hi,
> > >
> > > I am working on a multivariate linear regression of the
> form y = Ax.
> > >
> > > I am seeing a great dispersion of y w.r.t x. For example, the
> > > correlations between y and x are very small, even after using
some
> > > typical transformations like log, power.
> > >
> > > I tried with simple linear regression, robust regression
> and ace and
> > > avas package in R (or splus). I didn't see an improvement
> in the fit
> > > and predictions over simple linear regression. (I also
> tried this with
> > > transformed variables)
> > >
> > > I am sure that some of you came across such data. How did you
> > > deal with it?
> > >
> > > Linear regressions are good for the data like y = x +
> > > 0.01Normal(mu,sigma2) i.e. a small noise (data observed
> in a lab). But
> > > linear regressions are bad for large noise, like typical
> market (or
> > > survey) data.
> > >
> > > Thank you,
> > > Nagu
> > >
> > > ______________________________________________
> > > R-help at stat.math.ethz.ch mailing list
> > > https://stat.ethz.ch/mailman/listinfo/r-help
> > > PLEASE do read the posting guide!
> > > http://www.R-project.org/posting-guide.html
> > >
> >
> >
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide!
> http://www.R-project.org/posting-guide.html
>