Access

You are not currently logged in.

Access JSTOR through your library or other institution:

login

Log in through your institution.

If You Use a Screen Reader

This content is available through Read Online (Free) program, which relies on page scans. Since scans are not currently available to screen readers, please contact JSTOR User Support for access. We'll provide a PDF copy for your screen reader.

Statistical Significance Is Not a "Kosher Certificate" for Observed Effects: A Critical Analysis of the Two-Step Approach to the Evaluation of Empirical Results

Sorel Cahan
Educational Researcher
Vol. 29, No. 1 (Jan. - Feb., 2000), pp. 31-34
Stable URL: http://www.jstor.org/stable/1176588
Page Count: 4
  • Read Online (Free)
  • Download ($29.00)
  • Subscribe ($19.50)
  • Cite this Item
Since scans are not currently available to screen readers, please contact JSTOR User Support for access. We'll provide a PDF copy for your screen reader.
Statistical Significance Is Not a "Kosher Certificate" for Observed Effects: A Critical Analysis of the Two-Step Approach to the Evaluation of Empirical Results
Preview not available

Abstract

In order to prevent misleading conclusions based on spurious observed effects--especially seductively large ones--Robinson and Levin (1997) suggested a two-step approach to the reporting and evaluation of empirical results. According to the two-step model, the evaluation of the magnitude and substantive significance of obtained effects should be conditional upon their statistical significance: Authors should first indicate whether the observed effect is statistically improbable, and only if it is should they then indicate how large or important it is. The purpose of this paper is (a) to show that even though the underlying intention--to prevent unwarranted evaluation of spurious observed effects--is a laudable one, the two-step approach is inappropriate for this purpose, and (b) to reiterate the preferred approach, namely increased sample size and computation of confidence intervals.

Page Thumbnails

  • Thumbnail: Page 
31
    31
  • Thumbnail: Page 
32
    32
  • Thumbnail: Page 
33
    33
  • Thumbnail: Page 
34
    34