Access

You are not currently logged in.

Access JSTOR through your library or other institution:

login

Log in through your institution.

Journal Article

Statistical Modeling of Expert Ratings on Medical Treatment Appropriateness

John S. Uebersax
Journal of the American Statistical Association
Vol. 88, No. 422 (Jun., 1993), pp. 421-427
DOI: 10.2307/2290320
Stable URL: http://www.jstor.org/stable/2290320
Page Count: 7
Were these topics helpful?
See somethings inaccurate? Let us know!

Select the topics that are inaccurate.

Cancel
  • Download ($14.00)
  • Add to My Lists
  • Cite this Item
Statistical Modeling of Expert Ratings on Medical Treatment Appropriateness
Preview not available

Abstract

This article uses latent structure analysis to model ordered category ratings by multiple experts on the appropriateness of indications for the medical procedure carotid endarterectomy. The statistical method used is a form of located latent class analysis, which combines elements of latent class and latent trait analysis. It assumes that treatment indications fall into distinct latent classes, with each latent class corresponding to a different level of appropriateness. The appropriateness rating of a treatment indication by a rater is assumed determined by the latent class membership of the indication, rating category thresholds of the rater, and random measurement error. The located latent class model has two alternative forms: a normal ogive form, which derives from the assumption of normally distributed measurement error, and a logistic approximation to the normal form. The approach has the following advantages for the analysis of ordered category ratings by multiple experts: (1) it assesses whether different raters base ratings on the same or different criteria; (2) it assesses rater bias--the tendency of some raters to make higher or lower ratings than others; (3) it characterizes rater differences in rating category definitions; (4) it provides theoretically based methods for combining the ratings of different raters; and (5) it provides a description of the distribution of the latent trait. The data examined are appropriateness ratings on 848 indications for carotid endarterectomy made by nine medical experts. The located latent class approach provides unique insights concerning the data. It identifies what appears to be a set of clear nonindications for carotid endarterectomy, but a corresponding set of clear indications is not evident. The results indicate that all raters measured a common latent trait of treatment appropriateness, but that some measured the trait better than others. Rater differences in overall bias and rating category definitions are evident. Two methods are used to combine raters' ratings. One uses ratings to calculate a continuous appropriateness score for each indication. The other uses ratings to assign indications to discrete outcome categories, each corresponding to a specific level of appropriateness. The located latent class approach for ordered category measures has possible applications besides the analysis of expert ratings, such as item analysis. Potential extensions of the model are discussed.

Page Thumbnails

  • Thumbnail: Page 
421
    421
  • Thumbnail: Page 
422
    422
  • Thumbnail: Page 
423
    423
  • Thumbnail: Page 
424
    424
  • Thumbnail: Page 
425
    425
  • Thumbnail: Page 
426
    426
  • Thumbnail: Page 
427
    427