NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ799312
Record Type: Journal
Publication Date: 2008-Apr
Pages: 17
Abstractor: As Provided
Reference Count: 23
ISSN: ISSN-1530-5058
Objective Standard Setting for Judge-Mediated Examinations
Stone, Gregory Ethan; Beltyukova, Svetlana; Fox, Christine M.
International Journal of Testing, v8 n2 p180-196 Apr 2008
Judge-mediated examinations are defined as those for which expert evaluation (using rubrics) is required to determine correctness, completeness, and reasonability of test-taker responses. The use of multifaceted Rasch modeling has led to improvements in the reliability of scoring such examinations. The establishment of criterion-referenced standards for these assessments has, however, remained a labor-intensive practice. This article explores a method for incorporating the establishment of minimal competency within the measurement model itself. Two high-stakes testing boards participated in a controlled experiment comparing a multifaceted Rasch modeled approach to a traditional modified borderline approach often used for standard setting in judge-mediated situations. Criterion points from each were compared to assess differences and support validity. Results of the experiment suggest that modeling standard setting as part of the performance evaluation is an effective and efficient way to define minimal competence. (Contains 2 tables and 3 figures.)
Lawrence Erlbaum. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site:
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A