Access

You are not currently logged in.

Access your personal account or get JSTOR access through your library or other institution:

login

Log in to your personal account or through your institution.

If you need an accessible version of this item please contact JSTOR User Support

Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence

James O. Berger and Thomas Sellke
Journal of the American Statistical Association
Vol. 82, No. 397 (Mar., 1987), pp. 112-122
DOI: 10.2307/2289131
Stable URL: http://www.jstor.org/stable/2289131
Page Count: 11
  • Download ($14.00)
  • Cite this Item
If you need an accessible version of this item please contact JSTOR User Support
Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence
Preview not available

Abstract

The problem of testing a point null hypothesis (or a "small interval" null hypothesis) is considered. Of interest is the relationship between the P value (or observed significance level) and conditional and Bayesian measures of evidence against the null hypothesis. Although one might presume that a small P value indicates the presence of strong evidence against the null, such is not necessarily the case. Expanding on earlier work [especially Edwards, Lindman, and Savage (1963) and Dickey (1977)], it is shown that actual evidence against a null (as measured, say, by posterior probability or comparative likelihood) can differ by an order of magnitude from the P value. For instance, data that yield a P value of .05, when testing a normal mean, result in a posterior probability of the null of at least .30 for any objective prior distribution. ("Objective" here means that equal prior weight is given the two hypotheses and that the prior is symmetric and nonincreasing away from the null; other definitions of "objective" will be seen to yield qualitatively similar results.) The overall conclusion is that P values can be highly misleading measures of the evidence provided by the data against the null hypothesis.

Page Thumbnails

  • Thumbnail: Page 
112
    112
  • Thumbnail: Page 
113
    113
  • Thumbnail: Page 
114
    114
  • Thumbnail: Page 
115
    115
  • Thumbnail: Page 
116
    116
  • Thumbnail: Page 
117
    117
  • Thumbnail: Page 
118
    118
  • Thumbnail: Page 
119
    119
  • Thumbnail: Page 
120
    120
  • Thumbnail: Page 
121
    121
  • Thumbnail: Page 
122
    122