ERIC Number: ED442828
Record Type: RIE
Publication Date: 2000-Apr-15
The Consequences of Judges Making Advanced Estimates of Impact on a Cut Score.
Buckendahl, Chad; Impara, James C.; Giraud, Gerald; Irwin, Patrick M.
School districts and credentialing agencies use information gathered in standard setting studies to establish minimum passing scores (MPS) for a variety of purposes. These scores may be used to make decisions ranging from subject remediation to licensure. Multiple standard setting methods may be used to provide a range of scores to the policy-making entity. The independence of these methods is important to the validity of the score recommendations. This paper examines the potential for introducing bias into the standard setting process by asking panelists for their expectations of impact prior to their making item performance estimates as might be done if using methods recommended by G. Dillon (1996) or Angoff (W. Angoff) "corrections" as recommended by D. de Gruiter (1985) or W. Hofstee (1983). Mixed results were found from five standard setting applications conducted in a variety of arenas. Three studies were conducted in school districts where teachers served as panelists, and two studies were conducted in the context of certification examinations so that the tests ranged from low-stakes to very high-stakes. A total of 70 panelists participated in all 5 studies. (Contains 10 references.) (SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Note: Paper presented at the Annual Meeting of the American Educational Research Association (New Orleans, LA, April 24-28, 2000).