Access

You are not currently logged in.

Access your personal account or get JSTOR access through your library or other institution:

login

Log in to your personal account or through your institution.

Examining Rater Errors in the Assessment of Written Composition with a Many-Faceted Rasch Model

George Engelhard, Jr.
Journal of Educational Measurement
Vol. 31, No. 2 (Summer, 1994), pp. 93-112
Stable URL: http://www.jstor.org/stable/1435170
Page Count: 20
  • More info
  • Cite this Item
Examining Rater Errors in the Assessment of Written Composition with a Many-Faceted Rasch Model
Preview not available

Abstract

This study describes several categories of rater errors (rater severity, halo effect, central tendency, and restriction of range). Criteria are presented for evaluating the quality of ratings based on a many-faceted Rasch measurement (FACETS) model for analyzing judgments. A random sample of 264 compositions rated by 15 raters and a validity committee from the 1990 administration of the Eighth Grade Writing Test in Georgia is used to illustrate the model. The data suggest that there are significant differences in rater severity. Evidence of a halo effect is found for two raters who appear to be rating the compositions holistically rather than analytically. Approximately 80% of the ratings are in the two middle categories of the rating scale, indicating that the error of central tendency is present. Restriction of range is evident when the unadjusted raw score distribution is examined, although this rater error is less evident when adjusted estimates of writing competence are used.

Page Thumbnails

  • Thumbnail: Page 
93
    93
  • Thumbnail: Page 
94
    94
  • Thumbnail: Page 
95
    95
  • Thumbnail: Page 
96
    96
  • Thumbnail: Page 
97
    97
  • Thumbnail: Page 
98
    98
  • Thumbnail: Page 
99
    99
  • Thumbnail: Page 
100
    100
  • Thumbnail: Page 
101
    101
  • Thumbnail: Page 
102
    102
  • Thumbnail: Page 
103
    103
  • Thumbnail: Page 
[104]
    [104]
  • Thumbnail: Page 
105
    105
  • Thumbnail: Page 
106
    106
  • Thumbnail: Page 
107
    107
  • Thumbnail: Page 
108
    108
  • Thumbnail: Page 
[109]
    [109]
  • Thumbnail: Page 
110
    110
  • Thumbnail: Page 
111
    111
  • Thumbnail: Page 
112
    112