Results 1 to 8 of 8

Math Help - P-value: A formal definition

  1. #1
    Newbie
    Joined
    Jul 2011
    Posts
    15

    P-value (statistical significance): A formal definition

    What's a "p-value"?

    Many resources provide informal definitions. There seem to be two equivalent ones:

    1) The smallest level at which H_0 would be rejected.
    2) The probability that you would obtain evidence against H_0 which is at least as strong as that actually observed, if H_0 were true.

    I'm looking for a formal statement of both of these definitions (preferably resorting to little or no measure theory), as well as for a proof of their equivalence.

    Thanks.
    Last edited by Mishmish; July 9th 2011 at 11:44 PM.
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Flow Master
    mr fantastic's Avatar
    Joined
    Dec 2007
    From
    Zeitgeist
    Posts
    16,948
    Thanks
    5

    Re: P-value (statistical significance): A formal definition

    Quote Originally Posted by Mishmish View Post
    What's a "p-value"?

    Many resources provide informal definitions. There seem to be two equivalent ones:

    1) The smallest level at which H_0 would be rejected.
    2) The probability that you would obtain evidence against H_0 which is at least as strong as that actually observed, if H_0 were true.

    I'm looking for a formal statement of both of these definitions (preferably resorting to little or no measure theory), as well as for a proof of their equivalence.

    Thanks.
    Here is the typical definition given in most statistics books:

    The p-value is the probability of obtaining a test statistic at least as extreme as the one that was actually observed, assuming that the null hypothesis is true.
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Newbie
    Joined
    Jul 2011
    Posts
    15

    Re: P-value (statistical significance): A formal definition

    Quote Originally Posted by mr fantastic View Post
    Here is the typical definition given in most statistics books:

    The p-value is the probability of obtaining a test statistic at least as extreme as the one that was actually observed, assuming that the null hypothesis is true.
    Thanks, mr fantastic, but I'm not after the typical textbook definition since, unfortunately, the typical textbook falls short of defining this concept rigorously.

    Take, for instance, the informal definition you provided. What does "test statistic" mean (formally)? What does "at least as extreme as" mean (formally)?

    And how does this definition relate to the alternative definition I mentioned (#1), which can also be found in several "typical" textbooks (e.g. Wasserman's "All of Statistics" and Lehmann & Romano's "Hypothesis Testing") as well as in the link I provided?
    Follow Math Help Forum on Facebook and Google+

  4. #4
    Newbie
    Joined
    Jul 2011
    Posts
    15

    Re: P-value: A formal definition

    See attached file for my own formulation of said concept.
    Attached Files Attached Files
    Follow Math Help Forum on Facebook and Google+

  5. #5
    Guy
    Guy is offline
    Member
    Joined
    Mar 2010
    Posts
    98

    Re: P-value: A formal definition

    Unfortunately I don't have my copy of Casella and Berger on hand, but they give a more rigorous definition. It is something like, "a p-value is a statistic which has a Uniform(0, 1) distribution under the null hypothesis." I'm not sure how they handle composite nulls.
    Follow Math Help Forum on Facebook and Google+

  6. #6
    Newbie
    Joined
    Jul 2011
    Posts
    15

    Re: P-value: A formal definition

    Dear Guy, Thanks for the reference to Berger & Casella. While the Berger & Casella definition may be more rigorous than usual, it is, imo, not as intuitive as the two informal definitions used almost ubiquitously in textbooks and other pedagogical sources I have found on the web (see for instance the typical slide show i provided a link to in this thread's first message). It is these intuitions that I've attempted to formalize as well as to establish the equivalence between them.
    Follow Math Help Forum on Facebook and Google+

  7. #7
    Guy
    Guy is offline
    Member
    Joined
    Mar 2010
    Posts
    98

    Re: P-value: A formal definition

    Quote Originally Posted by Mishmish View Post
    Dear Guy, Thanks for the reference to Berger & Casella. While the Berger & Casella definition may be more rigorous than usual, it is, imo, not as intuitive as the two informal definitions used almost ubiquitously in textbooks and other pedagogical sources I have found on the web (see for instance the typical slide show i provided a link to in this thread's first message). It is these intuitions that I've attempted to formalize as well as to establish the equivalence between them.
    Maybe this will be food for thought then. I took a course from Casella, and the topic of p-values came up. He is someone who has thought about the use of p-values in the sense of (1), and has stated openly that he has no idea what (1) means from a formal standpoint. From the frequentist viewpoint, he can't make sense of it in terms of repeated experiments. Granted, I didn't get the impression that he had spent a particularly large amount of time on it. He is also not a frequentist.

    I think the definition of Casella and Burger actually lines up fairly well with how p-values get applied these days when developing new applications. We can work at the "level of test statistics" or the "level of p-values" and the work done at the p-value level makes heavy use of the fact that they are distributed as Uniform(0, 1) under the null hypothesis. Once you have this definition, you can go about showing how to derive p-values from tests that have the same size and optimality properties.
    Follow Math Help Forum on Facebook and Google+

  8. #8
    Newbie
    Joined
    Jul 2011
    Posts
    15

    Re: P-value: A formal definition

    Interesting. Thanks.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Formal definition of limits of sequences
    Posted in the Calculus Forum
    Replies: 0
    Last Post: September 11th 2010, 06:43 PM
  2. the formal definition of limit
    Posted in the Differential Geometry Forum
    Replies: 1
    Last Post: December 1st 2009, 06:18 PM
  3. the formal definition of limit
    Posted in the Differential Geometry Forum
    Replies: 0
    Last Post: November 29th 2009, 02:23 PM
  4. Formal Definition of Derivative
    Posted in the Calculus Forum
    Replies: 5
    Last Post: September 30th 2007, 10:42 AM
  5. Replies: 4
    Last Post: September 9th 2007, 08:42 PM

Search Tags


/mathhelpforum @mathhelpforum