NotesFAQContact Us
Search Tips
ERIC Number: ED563414
Record Type: Non-Journal
Publication Date: 2012-Apr-15
Pages: 22
Abstractor: As Provided
Reference Count: N/A
Using the Many-Facet Rasch Model to Evaluate Standard-Setting Judgments: Setting Performance Standards for Advanced Placement® Examinations
Kaliski, Pamela; Wind, Stefanie A.; Engelhard, George, Jr.; Morgan, Deanna; Plake, Barbara; Reshetar, Rosemary
College Board, Slides presented at the Annual Meeting of the American Educational Research Association (AERA) (Vancouver, BC, Apr 15, 2012)
The Many-Facet Rasch (MFR) Model is traditionally used to evaluate the quality of ratings on constructed response assessments; however, it can also be used to evaluate the quality of judgments from panel-based standard setting procedures. The current study illustrates the use of the MFR Model by examining the quality of ratings obtained from a standard setting employing the Multiple Yes-No (MYN) modified Angoff standard setting procedure to set the cuts on the Advanced Placement® (AP®) Environmental Science exam. The locations of the panelists, judged item difficulties, rounds, and performance standards using the MFR Model were examined. Results revealed evidence of acceptable quality of the ratings obtained from the panelists. Thus, the current study provides an important contribution of validity evidence for the AP Environmental Science standard setting, as well as an illustration of the utility of the MFR Model to evaluate standard setting judgments.
College Board. 250 Vesey Street, New York, NY 10281. Tel: 212-713-8000; e-mail:; Web site:
Publication Type: Non-Print Media; Reference Materials - General
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: College Board
Identifiers - Assessments and Surveys: Advanced Placement Examinations (CEEB)