R help - Aug 2016 - Proving (instead of rejecting) that two groups are actually equal

Hej R helpers

The standard in statistical hypothesis testing is to reject the null 
hypothesis that there is a difference between groups, i.e. to "prove" 
the alternative. However, failing to reject the null hypothesis does not 
prove it; its rejection just fails.

Now, as stated in the article "Unicorns do exist: a tutorial on 
"proving" the null hypothesis." by David L Streiner (Canadian
Journal of
Psychiatry, 48(11) 2003), we can define the null hypothesis to be that 
there IS a difference (exceeding a certain value, delta), the 
alternative hypothesis being that there is none (or it is at least 
smaller than delta). If the data now manages to reject the null 
hypothesis (of there being a difference exceeding delta), we can say 
with a certain probability that there is none.

Can I do this test in R? And if yes, any leads?

(In my actual dataset I deal with paired data.)

Best
    Dominik

Rejecting a null of "inequality" is the standard setup for equivalence
testing in medical contexts. Search on "equivalence testing in R" and
you will find what you need.

Cheers,
Bert


Bert Gunter

"The trouble with having an open mind is that people keep coming along
and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Wed, Aug 10, 2016 at 3:22 AM, Dominik Marti <dom at inik.ch>
wrote:
> Hej R helpers
>
> The standard in statistical hypothesis testing is to reject the null
> hypothesis that there is a difference between groups, i.e. to
"prove" the
> alternative. However, failing to reject the null hypothesis does not prove
> it; its rejection just fails.
>
> Now, as stated in the article "Unicorns do exist: a tutorial on
"proving"
> the null hypothesis." by David L Streiner (Canadian Journal of
Psychiatry,
> 48(11) 2003), we can define the null hypothesis to be that there IS a
> difference (exceeding a certain value, delta), the alternative hypothesis
> being that there is none (or it is at least smaller than delta). If the
data
> now manages to reject the null hypothesis (of there being a difference
> exceeding delta), we can say with a certain probability that there is none.
>
> Can I do this test in R? And if yes, any leads?
>
> (In my actual dataset I deal with paired data.)
>
> Best
>    Dominik
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

> On Aug 10, 2016, at 5:22 AM, Dominik Marti <dom at inik.ch> wrote:
> 
> Hej R helpers
> 
> The standard in statistical hypothesis testing is to reject the null
hypothesis that there is a difference between groups, i.e. to "prove"
the alternative. However, failing to reject the null hypothesis does not prove
it; its rejection just fails.
> 
> Now, as stated in the article "Unicorns do exist: a tutorial on
"proving" the null hypothesis." by David L Streiner (Canadian
Journal of Psychiatry, 48(11) 2003), we can define the null hypothesis to be
that there IS a difference (exceeding a certain value, delta), the alternative
hypothesis being that there is none (or it is at least smaller than delta). If
the data now manages to reject the null hypothesis (of there being a difference
exceeding delta), we can say with a certain probability that there is none.
> 
> Can I do this test in R? And if yes, any leads?
> 
> (In my actual dataset I deal with paired data.)
> 
> Best
>   Dominik

Bear in mind that we are not "proving" anything with statistics. There
is still a level of uncertainty in everything we do.

In the scenario above, you are, in essence, reversing the normal approach to
testing a null versus alternative hypothesis. The null, in this case, is that
there is a difference and the alternative being that there is none, within some
pre-defined, acceptable, margin.

In clinical studies, these are called "equivalence" studies or
"bioequivalence" studies, a subset of which are called
"non-inferiority" studies, which are one-sided versions. This is
typically done, for example, when testing a generic version of a drug versus the
pre-existing "brand name" version of the drug to demonstrate that they
have equivalent efficacy and safety profiles, within a clinically acceptable
range.

There is at least one R package that is relevant, conveniently called
"equivalence":

  https://cran.r-project.org/web/packages/equivalence/

that addresses these scenarios.

Regards,

Marc Schwartz