Access

You are not currently logged in.

Access JSTOR through your library or other institution:

login

Log in through your institution.

If You Use a Screen Reader

This content is available through Read Online (Free) program, which relies on page scans. Since scans are not currently available to screen readers, please contact JSTOR User Support for access. We'll provide a PDF copy for your screen reader.

Reflections on Statistical and Substantive Significance, with a Slice of Replication

Daniel H. Robinson and Joel R. Levin
Educational Researcher
Vol. 26, No. 5 (Jun. - Jul., 1997), pp. 21-26
Stable URL: http://www.jstor.org/stable/1176544
Page Count: 6
  • Read Online (Free)
  • Download ($29.00)
  • Subscribe ($19.50)
  • Cite this Item
Since scans are not currently available to screen readers, please contact JSTOR User Support for access. We'll provide a PDF copy for your screen reader.
Reflections on Statistical and Substantive Significance, with a Slice of Replication
Preview not available

Abstract

In this comment, we propose some modifications to Thompson's (1996) recent suggestions for AERA editorial policy on statistical significance testing. First, we discuss the potential problems, both procedural and conceptual, that could arise from professional journals mandating the addition of the modifier "statistically" to "significant." If language remains an issue of consequence, however, findings stemming from rejected null hypotheses could be termed "nonchance" to communicate simply and more precisely what it is that a statistical test reveals. Second, although asking authors to provide explicit effect-size information is both sensible and useful, we illustrate how effect sizes (like p values) can be misinterpreted and misused. Finally, in accord with Thompson, we argue that greater attention to replication should be encouraged in educational research. At the same time, we believe that internal replication analyses do not represent adequate substitutes for (or even close approximations to) external replication studies based on new, independent participants. With journals welcoming both constructed and literal replication studies, consumers of the educational research literature can concentrate less on the limited believability of one-time findings and more on the enhanced believability of repeatable and generalizable ones.

Page Thumbnails

  • Thumbnail: Page 
21
    21
  • Thumbnail: Page 
22
    22
  • Thumbnail: Page 
23
    23
  • Thumbnail: Page 
24
    24
  • Thumbnail: Page 
25
    25
  • Thumbnail: Page 
26
    26