Dear all; A very basic question. I have the following data: ************************************************************************************ A <- 1/1000*c(347,328,129,122,18,57,105,188,57,257,53,108,336,163, 62,112,334,249,45,244,211,175,174,26,375,346,153,32, 89,32,358,202,123,131,88,36,30,67,96,135,219,122, 89,117,86,169,179,54,48,40,54,568,664,277,91,290, 116,80,107,401,225,517,90,133,36,50,174,103,192,150, 225,29,80,199,55,258,97,109,137,90,236,109,204,160, 95,54,50,78,98,141,508,144,434,100,37,22,304,175, 72,71,111,60,212,73,50,92,70,148,28,63,46,85, 111,67,234,65,92,59,118,202,21,17,95,86,296,45, 139,32,21,70,185,172,151,129,42,14,13,75,303,119, 128,106,224,241,112,395,78,89,247,122,212,61,165,30, 65,261,415,159,316,182,141,184,124,223,39,141,103,149, 104,71,259,86,85,214,96,246,306,11,129) B <- 1/1000*c(351,313,130,119,17,50,105,181,58,255,51,98,335,162, 60,108,325,240,44,242,208,168,170,27,356,341,150,31, 85,29,363,185,124,131,85,35,27,63,92,147,217,117, 87,119,81,161,178,53,45,38,50,581,661,254,87,281, 110,76,100,401,220,507,94,123,36,47,154,99,184,146, 232,26,77,193,53,264,94,110,128,87,231,110,195,156, 95,51,50,75,93,134,519,139,435,96,37,21,293,169, 70,80,104,64,210,70,48,88,67,140,26,52,45,90, 106,63,219,62,91,56,113,187,18,14,95,86,284,39, 132,31,22,69,181,167,150,117,42,14,11,73,303,109, 129,106,227,249,111,409,71,88,256,120,200,60,159,27, 63,268,389,150,311,175,136,171,116,220,30,145,95,148, 102,70,251,88,83,199,94,245,305,9,129) ************************************************************************************ plot(A,B) abline(0,1) At a glance, the data look very similar. Data A and B are two measurements of the same variable but using different devices (on a same set of subjects). Thus, I thought that a paired t-test could be appropriate to check if the diff between measurement devices = 0. t.test(A-B) ************************************************************************************ One Sample t-test data: A - B t = 7.6276, df = 178, p-value = 1.387e-12 alternative hypothesis: true mean is not equal to 0 95 percent confidence interval: 0.002451622 0.004162903 sample estimates: mean of x 0.003307263 ************************************************************************************ The mean diff is 0.0033 but the p-value indicates a strong evidence to reject H0. I was expecting to find no differences so I'm wondering whether the t-test is the appropriate test to use. I'll appreciate any comments or suggestions. BR, PM
Yes, in over 3/4s of the data points A is > B? which suggests the A measure is reading higher than the B measuring system. length(A[A>B])/length(A) On 20 Sep 2011, at 6:46 PM, Pedro Mardones wrote:> Dear all; > > A very basic question. I have the following data: > > ************************************************************************************ > > A <- 1/1000*c(347,328,129,122,18,57,105,188,57,257,53,108,336,163, > 62,112,334,249,45,244,211,175,174,26,375,346,153,32, > 89,32,358,202,123,131,88,36,30,67,96,135,219,122, > 89,117,86,169,179,54,48,40,54,568,664,277,91,290, > 116,80,107,401,225,517,90,133,36,50,174,103,192,150, > 225,29,80,199,55,258,97,109,137,90,236,109,204,160, > 95,54,50,78,98,141,508,144,434,100,37,22,304,175, > 72,71,111,60,212,73,50,92,70,148,28,63,46,85, > 111,67,234,65,92,59,118,202,21,17,95,86,296,45, > 139,32,21,70,185,172,151,129,42,14,13,75,303,119, > 128,106,224,241,112,395,78,89,247,122,212,61,165,30, > 65,261,415,159,316,182,141,184,124,223,39,141,103,149, > 104,71,259,86,85,214,96,246,306,11,129) > > B <- 1/1000*c(351,313,130,119,17,50,105,181,58,255,51,98,335,162, > 60,108,325,240,44,242,208,168,170,27,356,341,150,31, > 85,29,363,185,124,131,85,35,27,63,92,147,217,117, > 87,119,81,161,178,53,45,38,50,581,661,254,87,281, > 110,76,100,401,220,507,94,123,36,47,154,99,184,146, > 232,26,77,193,53,264,94,110,128,87,231,110,195,156, > 95,51,50,75,93,134,519,139,435,96,37,21,293,169, > 70,80,104,64,210,70,48,88,67,140,26,52,45,90, > 106,63,219,62,91,56,113,187,18,14,95,86,284,39, > 132,31,22,69,181,167,150,117,42,14,11,73,303,109, > 129,106,227,249,111,409,71,88,256,120,200,60,159,27, > 63,268,389,150,311,175,136,171,116,220,30,145,95,148, > 102,70,251,88,83,199,94,245,305,9,129) > > ************************************************************************************ > > plot(A,B) > abline(0,1) > > At a glance, the data look very similar. Data A and B are two > measurements of the same variable but using different devices (on a > same set of subjects). Thus, I thought that a paired t-test could be > appropriate to check if the diff between measurement devices = 0. > > t.test(A-B) > > ************************************************************************************ > > One Sample t-test > > data: A - B > t = 7.6276, df = 178, p-value = 1.387e-12 > alternative hypothesis: true mean is not equal to 0 > 95 percent confidence interval: > 0.002451622 0.004162903 > sample estimates: > mean of x > 0.003307263 > > ************************************************************************************ > The mean diff is 0.0033 but the p-value indicates a strong evidence to > reject H0. > > I was expecting to find no differences so I'm wondering whether the > t-test is the appropriate test to use. I'll appreciate any comments or > suggestions. > > BR, > PM > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.
On Sep 20, 2011, at 12:46 PM, Pedro Mardones wrote:> Dear all; > > A very basic question. I have the following data: > > ************************************************************************************ > > A <- 1/1000*c(347,328,129,122,18,57,105,188,57,257,53,108,336,163, > 62,112,334,249,45,244,211,175,174,26,375,346,153,32, > 89,32,358,202,123,131,88,36,30,67,96,135,219,122, > 89,117,86,169,179,54,48,40,54,568,664,277,91,290, > 116,80,107,401,225,517,90,133,36,50,174,103,192,150, > 225,29,80,199,55,258,97,109,137,90,236,109,204,160, > 95,54,50,78,98,141,508,144,434,100,37,22,304,175, > 72,71,111,60,212,73,50,92,70,148,28,63,46,85, > 111,67,234,65,92,59,118,202,21,17,95,86,296,45, > 139,32,21,70,185,172,151,129,42,14,13,75,303,119, > 128,106,224,241,112,395,78,89,247,122,212,61,165,30, > 65,261,415,159,316,182,141,184,124,223,39,141,103,149, > 104,71,259,86,85,214,96,246,306,11,129) > > B <- 1/1000*c(351,313,130,119,17,50,105,181,58,255,51,98,335,162, > 60,108,325,240,44,242,208,168,170,27,356,341,150,31, > 85,29,363,185,124,131,85,35,27,63,92,147,217,117, > 87,119,81,161,178,53,45,38,50,581,661,254,87,281, > 110,76,100,401,220,507,94,123,36,47,154,99,184,146, > 232,26,77,193,53,264,94,110,128,87,231,110,195,156, > 95,51,50,75,93,134,519,139,435,96,37,21,293,169, > 70,80,104,64,210,70,48,88,67,140,26,52,45,90, > 106,63,219,62,91,56,113,187,18,14,95,86,284,39, > 132,31,22,69,181,167,150,117,42,14,11,73,303,109, > 129,106,227,249,111,409,71,88,256,120,200,60,159,27, > 63,268,389,150,311,175,136,171,116,220,30,145,95,148, > 102,70,251,88,83,199,94,245,305,9,129) > > ************************************************************************************ > > plot(A,B) > abline(0,1) > > At a glance, the data look very similar. Data A and B are two > measurements of the same variable but using different devices (on a > same set of subjects). Thus, I thought that a paired t-test could be > appropriate to check if the diff between measurement devices = 0. > > t.test(A-B) > > ************************************************************************************ > > One Sample t-test > > data: A - B > t = 7.6276, df = 178, p-value = 1.387e-12 > alternative hypothesis: true mean is not equal to 0 > 95 percent confidence interval: > 0.002451622 0.004162903 > sample estimates: > mean of x > 0.003307263 > > ************************************************************************************ > The mean diff is 0.0033 but the p-value indicates a strong evidence to > reject H0. > > I was expecting to find no differences so I'm wondering whether the > t-test is the appropriate test to use. I'll appreciate any comments or > suggestions. > > BR, > PMYou should look at: http://www-users.york.ac.uk/~mb55/meas/meas.htm which provides insights into how to compare measurement methods. You might start with the FAQ. The paired t-test does not tell you if there are systematic differences across the range of measures. HTH, Marc Schwartz
Pedro Mardones wrote on 09/20/2011 12:46:54 PM:> > Dear all; > > A very basic question. I have the following data: > >************************************************************************************> > A <- 1/1000*c(347,328,129,122,18,57,105,188,57,257,53,108,336,163, > 62,112,334,249,45,244,211,175,174,26,375,346,153,32, > 89,32,358,202,123,131,88,36,30,67,96,135,219,122, > 89,117,86,169,179,54,48,40,54,568,664,277,91,290, > 116,80,107,401,225,517,90,133,36,50,174,103,192,150, > 225,29,80,199,55,258,97,109,137,90,236,109,204,160, > 95,54,50,78,98,141,508,144,434,100,37,22,304,175, > 72,71,111,60,212,73,50,92,70,148,28,63,46,85, > 111,67,234,65,92,59,118,202,21,17,95,86,296,45, > 139,32,21,70,185,172,151,129,42,14,13,75,303,119, > 128,106,224,241,112,395,78,89,247,122,212,61,165,30, > 65,261,415,159,316,182,141,184,124,223,39,141,103,149, > 104,71,259,86,85,214,96,246,306,11,129) > > B <- 1/1000*c(351,313,130,119,17,50,105,181,58,255,51,98,335,162, > 60,108,325,240,44,242,208,168,170,27,356,341,150,31, > 85,29,363,185,124,131,85,35,27,63,92,147,217,117, > 87,119,81,161,178,53,45,38,50,581,661,254,87,281, > 110,76,100,401,220,507,94,123,36,47,154,99,184,146, > 232,26,77,193,53,264,94,110,128,87,231,110,195,156, > 95,51,50,75,93,134,519,139,435,96,37,21,293,169, > 70,80,104,64,210,70,48,88,67,140,26,52,45,90, > 106,63,219,62,91,56,113,187,18,14,95,86,284,39, > 132,31,22,69,181,167,150,117,42,14,11,73,303,109, > 129,106,227,249,111,409,71,88,256,120,200,60,159,27, > 63,268,389,150,311,175,136,171,116,220,30,145,95,148, > 102,70,251,88,83,199,94,245,305,9,129) > >************************************************************************************> > plot(A,B) > abline(0,1) > > At a glance, the data look very similar. Data A and B are two > measurements of the same variable but using different devices (on a > same set of subjects). Thus, I thought that a paired t-test could be > appropriate to check if the diff between measurement devices = 0. > > t.test(A-B) > >************************************************************************************> > One Sample t-test > > data: A - B > t = 7.6276, df = 178, p-value = 1.387e-12 > alternative hypothesis: true mean is not equal to 0 > 95 percent confidence interval: > 0.002451622 0.004162903 > sample estimates: > mean of x > 0.003307263 > >************************************************************************************> The mean diff is 0.0033 but the p-value indicates a strong evidence to > reject H0. > > I was expecting to find no differences so I'm wondering whether the > t-test is the appropriate test to use. I'll appreciate any comments or > suggestions. > > BR, > PMThe paired t-test could very well be an appropriate test to use, depending on the design of the study. If you look at the distribution of difference between A and B it certainly looks like the mean of that distribution is not 0: hist(A-B, nclass=50) abline(v=0, lwd=2) Perhaps what you need to think about is the difference between statistical significance and "practical" significance. Is a difference of about 2% between measurement devices "important"? Jean [[alternative HTML version deleted]]
It looks like Bland-Altman procedures would be appropriate for this project (http://en.wikipedia.org/wiki/Bland-Altman_plot). -----Original Message----- From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On Behalf Of Pedro Mardones Sent: Tuesday, September 20, 2011 11:47 AM To: R-help at r-project.org Subject: [R] help in interpreting paired t-test Dear all; A very basic question. I have the following data: ************************************************************************************ A <- 1/1000*c(347,328,129,122,18,57,105,188,57,257,53,108,336,163, 62,112,334,249,45,244,211,175,174,26,375,346,153,32, 89,32,358,202,123,131,88,36,30,67,96,135,219,122, 89,117,86,169,179,54,48,40,54,568,664,277,91,290, 116,80,107,401,225,517,90,133,36,50,174,103,192,150, 225,29,80,199,55,258,97,109,137,90,236,109,204,160, 95,54,50,78,98,141,508,144,434,100,37,22,304,175, 72,71,111,60,212,73,50,92,70,148,28,63,46,85, 111,67,234,65,92,59,118,202,21,17,95,86,296,45, 139,32,21,70,185,172,151,129,42,14,13,75,303,119, 128,106,224,241,112,395,78,89,247,122,212,61,165,30, 65,261,415,159,316,182,141,184,124,223,39,141,103,149, 104,71,259,86,85,214,96,246,306,11,129) B <- 1/1000*c(351,313,130,119,17,50,105,181,58,255,51,98,335,162, 60,108,325,240,44,242,208,168,170,27,356,341,150,31, 85,29,363,185,124,131,85,35,27,63,92,147,217,117, 87,119,81,161,178,53,45,38,50,581,661,254,87,281, 110,76,100,401,220,507,94,123,36,47,154,99,184,146, 232,26,77,193,53,264,94,110,128,87,231,110,195,156, 95,51,50,75,93,134,519,139,435,96,37,21,293,169, 70,80,104,64,210,70,48,88,67,140,26,52,45,90, 106,63,219,62,91,56,113,187,18,14,95,86,284,39, 132,31,22,69,181,167,150,117,42,14,11,73,303,109, 129,106,227,249,111,409,71,88,256,120,200,60,159,27, 63,268,389,150,311,175,136,171,116,220,30,145,95,148, 102,70,251,88,83,199,94,245,305,9,129) ************************************************************************************ plot(A,B) abline(0,1) At a glance, the data look very similar. Data A and B are two measurements of the same variable but using different devices (on a same set of subjects). Thus, I thought that a paired t-test could be appropriate to check if the diff between measurement devices = 0. t.test(A-B) ************************************************************************************ One Sample t-test data: A - B t = 7.6276, df = 178, p-value = 1.387e-12 alternative hypothesis: true mean is not equal to 0 95 percent confidence interval: 0.002451622 0.004162903 sample estimates: mean of x 0.003307263 ************************************************************************************ The mean diff is 0.0033 but the p-value indicates a strong evidence to reject H0. I was expecting to find no differences so I'm wondering whether the t-test is the appropriate test to use. I'll appreciate any comments or suggestions. BR, PM ______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
> cor(A, B)[1] 0.9986861 The data are very, very highly correlated. The higher the correlation, the greater the power of the t-test to detect the same difference between the means. Jeremy On 20 September 2011 10:46, Pedro Mardones <mardones.p at gmail.com> wrote:> Dear all; > > A very basic question. I have the following data: > > ************************************************************************************ > > A <- 1/1000*c(347,328,129,122,18,57,105,188,57,257,53,108,336,163, > 62,112,334,249,45,244,211,175,174,26,375,346,153,32, > 89,32,358,202,123,131,88,36,30,67,96,135,219,122, > 89,117,86,169,179,54,48,40,54,568,664,277,91,290, > 116,80,107,401,225,517,90,133,36,50,174,103,192,150, > 225,29,80,199,55,258,97,109,137,90,236,109,204,160, > 95,54,50,78,98,141,508,144,434,100,37,22,304,175, > 72,71,111,60,212,73,50,92,70,148,28,63,46,85, > 111,67,234,65,92,59,118,202,21,17,95,86,296,45, > 139,32,21,70,185,172,151,129,42,14,13,75,303,119, > 128,106,224,241,112,395,78,89,247,122,212,61,165,30, > 65,261,415,159,316,182,141,184,124,223,39,141,103,149, > 104,71,259,86,85,214,96,246,306,11,129) > > B <- 1/1000*c(351,313,130,119,17,50,105,181,58,255,51,98,335,162, > 60,108,325,240,44,242,208,168,170,27,356,341,150,31, > 85,29,363,185,124,131,85,35,27,63,92,147,217,117, > 87,119,81,161,178,53,45,38,50,581,661,254,87,281, > 110,76,100,401,220,507,94,123,36,47,154,99,184,146, > 232,26,77,193,53,264,94,110,128,87,231,110,195,156, > 95,51,50,75,93,134,519,139,435,96,37,21,293,169, > 70,80,104,64,210,70,48,88,67,140,26,52,45,90, > 106,63,219,62,91,56,113,187,18,14,95,86,284,39, > 132,31,22,69,181,167,150,117,42,14,11,73,303,109, > 129,106,227,249,111,409,71,88,256,120,200,60,159,27, > 63,268,389,150,311,175,136,171,116,220,30,145,95,148, > 102,70,251,88,83,199,94,245,305,9,129) > > ************************************************************************************ > > plot(A,B) > abline(0,1) > > At a glance, the data look very similar. Data A and B are two > measurements of the same variable but using different devices (on a > same set of subjects). Thus, I thought that a paired t-test could be > appropriate to check if the diff between measurement devices = 0. > > t.test(A-B) > > ************************************************************************************ > > One Sample t-test > > data: ?A - B > t = 7.6276, df = 178, p-value = 1.387e-12 > alternative hypothesis: true mean is not equal to 0 > 95 percent confidence interval: > ?0.002451622 0.004162903 > sample estimates: > ?mean of x > 0.003307263 > > ************************************************************************************ > The mean diff is 0.0033 but the p-value indicates a strong evidence to > reject H0. > > I was expecting to find no differences so I'm wondering whether the > t-test is the appropriate test to use. I'll appreciate any comments or > suggestions. > > BR, > PM > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >