Access

You are not currently logged in.

Access your personal account or get JSTOR access through your library or other institution:

login

Log in to your personal account or through your institution.

A Better Confidence Interval for Kappa (κ) on Measuring Agreement between Two Raters with Binary Outcomes

J. Jack Lee and Z. Nora Tu
Journal of Computational and Graphical Statistics
Vol. 3, No. 3 (Sep., 1994), pp. 301-321
DOI: 10.2307/1390914
Stable URL: http://www.jstor.org/stable/1390914
Page Count: 21
  • Download ($14.00)
  • Cite this Item
A Better Confidence Interval for Kappa (κ) on Measuring Agreement between Two Raters with Binary Outcomes
Preview not available

Abstract

Although the kappa statistic is widely used in measuring interrater agreement, it is known that the standard confidence interval estimation behaves poorly in small samples and for nonzero kappas. Efforts have been made to improve the estimation through transformation and Edgeworth expansion (Flack 1987). The results remain unsatisfactory when kappa is far from 0, however, even with the sample size as large as 100. In this article we reparameterize the kappa statistic to reveal its relationship with the marginal probability of agreement. The reparameterization not only gives a more meaningful interpretation of kappa but also clearly demonstrates that the range of kappa depends on the marginal probabilities. Various two- and three-dimensional plots are shown to illustrate the relationship among these parameters. The immediate application is to construct a new confidence interval based on the profile variance and reparameterization. Extensive simulation studies show that the new confidence interval performs very well in almost all parameter settings even when other methods fail.

Page Thumbnails

  • Thumbnail: Page 
301
    301
  • Thumbnail: Page 
302
    302
  • Thumbnail: Page 
303
    303
  • Thumbnail: Page 
304
    304
  • Thumbnail: Page 
305
    305
  • Thumbnail: Page 
306
    306
  • Thumbnail: Page 
307
    307
  • Thumbnail: Page 
308
    308
  • Thumbnail: Page 
309
    309
  • Thumbnail: Page 
310
    310
  • Thumbnail: Page 
311
    311
  • Thumbnail: Page 
312
    312
  • Thumbnail: Page 
313
    313
  • Thumbnail: Page 
314
    314
  • Thumbnail: Page 
315
    315
  • Thumbnail: Page 
316
    316
  • Thumbnail: Page 
317
    317
  • Thumbnail: Page 
318
    318
  • Thumbnail: Page 
319
    319
  • Thumbnail: Page 
320
    320
  • Thumbnail: Page 
321
    321