ERIC Number: EJ940998
Record Type: Journal
Publication Date: 2011
Abstractor: As Provided
Reference Count: 41
A Hierarchical Rater Model for Constructed Responses, with a Signal Detection Rater Model
DeCarlo, Lawrence T.; Kim, YoungKoung; Johnson, Matthew S.
Journal of Educational Measurement, v48 n3 p333-356 Fall 2011
The hierarchical rater model (HRM) recognizes the hierarchical structure of data that arises when raters score constructed response items. In this approach, raters' scores are not viewed as being direct indicators of examinee proficiency but rather as indicators of essay quality; the (latent categorical) quality of an examinee's essay in turn serves as an indicator of the examinee's proficiency, thus yielding a hierarchical structure. Here it is shown that a latent class model motivated by signal detection theory (SDT) is a natural candidate for the first level of the HRM, the rater model. The latent class SDT model provides measures of rater precision and various rater effects, above and beyond simply severity or leniency. The HRM-SDT model is applied to data from a large-scale assessment and is shown to provide a useful summary of various aspects of the raters' performance.
Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: firstname.lastname@example.org; Web site: http://www.wiley.com/WileyCDA/
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Authoring Institution: N/A