R help - May 2008 - Simulating the p-value of a test statistic

Dear Friends,


I want to know if any one could give me help on this. 
Suppose you want to simulate the p-value of a test statistic 
but you do not know how the distribution of the test 
statistic, how can one simulate the p-value.

It will be helpful to get an example code in R on this

Respectfully Submitted,
Clement

ampad1cb at cmich.edu wrote:
> Dear Friends,
>
>
> I want to know if any one could give me help on this. 
> Suppose you want to simulate the p-value of a test statistic 
> but you do not know how the distribution of the test 
> statistic, how can one simulate the p-value.
>
> It will be helpful to get an example code in R on this'
>   
If you want R code, you need to give more background.  Here's R code to 
simulate the p-value under the null:

p <- runif(1000)

Not very interesting.  It would be more interesting if we knew the context.

Duncan Murdoch

hi: you simulate the distribution of the test statistic by simulating 
the hypothesis test over and over and calculating the test statistic 
each time.
the details of how you simulate the hypothesis test depends on the 
particular problem you are dealing with but you didn't describe that.
if you describe it in more detail, maybe someone can give a more 
detailed response.




On Sat, May 24, 2008 at  7:04 PM, ampad1cb at cmich.edu wrote:
> Dear Friends,
>
>
> I want to know if any one could give me help on this. Suppose you want 
> to simulate the p-value of a test statistic but you do not know how 
> the distribution of the test statistic, how can one simulate the 
> p-value.
>
> It will be helpful to get an example code in R on this
>
> Respectfully Submitted,
> Clement
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide 
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

ampad1cb at cmich.edu wrote:
> The context is "goodness-of-fit techniques"
>
> Suppose I want to know whether a sample of data comes from 
> say an exponential distribution or not.
>
> I have a test statistic T, but I do not know its cummulative 
> distribution.
>
> Suppose an observed value of T is t
>
> How do I find the p-value in this case by simulation?
>
> that is how do I find P(T>t)
>   
Simulate 1000 (or 100000) values of T for which the null is true, and 
see where t falls in that distribution.

Duncan Murdoch

i think you should resend your question to the list (but this is 
sounding more and more like a homework problem so i don't know if you 
will get much response.  all i'll say is that P(T > t ) = 1 - P( T < t
). But P( T < t ) is the cumulative distribution of T ( and any random 
variable  ) is
uniformly distributed ) so you can generate uniform random variables ( 
as Duncan mentioned ) but i'm unsure what you do from there
with these uniform RV's ? Let me cc the R-help group ( hopefully you 
don't mind ) because i'm interested myself now that you brought it up.




On Sat, May 24, 2008 at  9:52 PM, ampad1cb at cmich.edu wrote:
> Dear Friend,
>
> The problem is this suppose you have a test statistic T and you dont 
> know its distribution, suppose t is an observed value of T. How can I 
> find P(T>t) by simulation, that is the p-value of the test statistic?
>
> -Clement

Thanks Duncan. The person didn't mention the exponential in their 
private email to me so I was pretty confused. Now, I'm unconfused.



On Sat, May 24, 2008 at 10:02 PM, Duncan Murdoch wrote:
> ampad1cb at cmich.edu wrote:
>> The context is "goodness-of-fit techniques"
>>
>> Suppose I want to know whether a sample of data comes from say an 
>> exponential distribution or not.
>>
>> I have a test statistic T, but I do not know its cummulative 
>> distribution.
>>
>> Suppose an observed value of T is t
>>
>> How do I find the p-value in this case by simulation?
>>
>> that is how do I find P(T>t)
>>
>
> Simulate 1000 (or 100000) values of T for which the null is true, and 
> see where t falls in that distribution.
>
> Duncan Murdoch
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide 
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.