R help - Jun 2020 - how to combine uncertainty and weighting in spearman correlation?

Hello everyone

At the moment I put a lot of attention in the uncertainty of my analyzes. I want
to do a spearman correlation that takes into account the uncertainty in my
observations and has weighting.

uncertainty of observations: I came across this excellent blog that proposes a
bootstrap function:
https://www.r-bloggers.com/finding-correlations-in-data-with-uncertainty/

weighted: I do weighted correlations with the wCorr package.

Now I want to combine both approaches in one approach for a final analysis. How
would you do that?

Thanks for the help!

Frederik Feys
PhD Medical Sciences
Onafhankelijk Methodoloog
https://www.researchgate.net/profile/Frederik_Feys
+32488020010







	[[alternative HTML version deleted]]

Hi Frederick,

I glanced at the webpage you've linked.
(But only the top three snippets).

This is what I would call the sum of random variables.
(X, Y) = (X1, X1) + (X2, Y2) + ... + (Xn, Yn)

The example makes the mistake of assuming that the Xs are normally
distributed, and each of the Ys are from exactly the same uniform
distribution.
By "combine"-ing both approaches, are you wanting to weight each pair?

w1(X1, X1) + w2(X2, Y2) + ... + wn(Xn, Yn)

I note that you haven't told us much about your data.
There may be an easier way of doing things...


On Mon, Jun 22, 2020 at 1:53 AM Frederik Feys <frefeys at gmail.com>
wrote:
>
> Hello everyone
>
> At the moment I put a lot of attention in the uncertainty of my analyzes. I
want to do a spearman correlation that takes into account the uncertainty in my
observations and has weighting.
>
> uncertainty of observations: I came across this excellent blog that
proposes a bootstrap function:
https://www.r-bloggers.com/finding-correlations-in-data-with-uncertainty/
>
> weighted: I do weighted correlations with the wCorr package.
>
> Now I want to combine both approaches in one approach for a final analysis.
How would you do that?
>
> Thanks for the help!
>
> Frederik Feys
> PhD Medical Sciences
> Onafhankelijk Methodoloog
> https://www.researchgate.net/profile/Frederik_Feys
> +32488020010
>
>
>
>
>
>
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

Just realised the above notation may be a bit misleading.
Because I was thinking in terms of simulated data.


On Mon, Jun 22, 2020 at 10:00 AM Abby Spurdle <spurdle.a at gmail.com>
wrote:
>
> Hi Frederick,
>
> I glanced at the webpage you've linked.
> (But only the top three snippets).
>
> This is what I would call the sum of random variables.
> (X, Y) = (X1, X1) + (X2, Y2) + ... + (Xn, Yn)
>
> The example makes the mistake of assuming that the Xs are normally
> distributed, and each of the Ys are from exactly the same uniform
> distribution.
> By "combine"-ing both approaches, are you wanting to weight each
pair?
>
> w1(X1, X1) + w2(X2, Y2) + ... + wn(Xn, Yn)
>
> I note that you haven't told us much about your data.
> There may be an easier way of doing things...
>
>
> On Mon, Jun 22, 2020 at 1:53 AM Frederik Feys <frefeys at gmail.com>
wrote:
> >
> > Hello everyone
> >
> > At the moment I put a lot of attention in the uncertainty of my
analyzes. I want to do a spearman correlation that takes into account the
uncertainty in my observations and has weighting.
> >
> > uncertainty of observations: I came across this excellent blog that
proposes a bootstrap function:
https://www.r-bloggers.com/finding-correlations-in-data-with-uncertainty/
> >
> > weighted: I do weighted correlations with the wCorr package.
> >
> > Now I want to combine both approaches in one approach for a final
analysis. How would you do that?
> >
> > Thanks for the help!
> >
> > Frederik Feys
> > PhD Medical Sciences
> > Onafhankelijk Methodoloog
> > https://www.researchgate.net/profile/Frederik_Feys
> > +32488020010
> >
> >
> >
> >
> >
> >
> >
> >         [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.

I need to fix my mistakes, from earlier this morning.
The sums should be over densities, so:

fh (X, Y) = [fh1 (X1, X1) + fh2 (X2, Y2) + ... + fhn (Xn, Yn)] / n

fh (X, Y) = w1*fh1 (X1, X1) + w2*fh2 (X2, Y2) + ... + wn*fhn (Xn, Yn)

    assuming the weights sum to 1

If simulated data is used, then the expressions above can be replaced
with the union of multiple (sub)samples.
Then an estimate/inference (say correlation) can be computed from one
or more combined samples.

Sorry, for triple posting.


On Mon, Jun 22, 2020 at 10:00 AM Abby Spurdle <spurdle.a at gmail.com>
wrote:
>
> Hi Frederick,
>
> I glanced at the webpage you've linked.
> (But only the top three snippets).
>
> This is what I would call the sum of random variables.
> (X, Y) = (X1, X1) + (X2, Y2) + ... + (Xn, Yn)
>
> The example makes the mistake of assuming that the Xs are normally
> distributed, and each of the Ys are from exactly the same uniform
> distribution.
> By "combine"-ing both approaches, are you wanting to weight each
pair?
>
> w1(X1, X1) + w2(X2, Y2) + ... + wn(Xn, Yn)
>
> I note that you haven't told us much about your data.
> There may be an easier way of doing things...
>
>
> On Mon, Jun 22, 2020 at 1:53 AM Frederik Feys <frefeys at gmail.com>
wrote:
> >
> > Hello everyone
> >
> > At the moment I put a lot of attention in the uncertainty of my
analyzes. I want to do a spearman correlation that takes into account the
uncertainty in my observations and has weighting.
> >
> > uncertainty of observations: I came across this excellent blog that
proposes a bootstrap function:
https://www.r-bloggers.com/finding-correlations-in-data-with-uncertainty/
> >
> > weighted: I do weighted correlations with the wCorr package.
> >
> > Now I want to combine both approaches in one approach for a final
analysis. How would you do that?
> >
> > Thanks for the help!
> >
> > Frederik Feys
> > PhD Medical Sciences
> > Onafhankelijk Methodoloog
> > https://www.researchgate.net/profile/Frederik_Feys
> > +32488020010
> >
> >
> >
> >
> >
> >
> >
> >         [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.