ERIC Number: ED294927
Record Type: RIE
Publication Date: 1988-Apr
Reference Count: 0
A Critique of Difficulty Estimation Methodologies in the Setting of Cut Points and a Discussion of an Alternative Methodology: The Direct Standard Setting Method.
Schoon, Craig G.; And Others
The determination of appropriate cut scores is a critical step in the development of licensing and certification examinations. Passing point methodologies based on the estimation of item difficulties are underlain by the estimation of the probability of a correct response to items by a hypothetically minimally competent candidate. The Angoff technique is typical of such methods. The use of such constructs as "minimally competent" and "probability of a correct response," however, in setting passing points is problematical. Little evidence exists concerning the utility of these constructs, and it is difficult to provide them with operational definitions or validate them against external criteria. As an alternative to these methods, a Direct Standard Setting Method has been developed upon the following bases: (1) emphasis on judges' estimation of required level of candidates' knowledge rather than probability of correct response; (2) development of a standard that provides the definition of the performance necessary for examinees to be certified; (3) an understanding by the judges that their ratings of questions will be translated into performance standards; and (4) a methodology that is easy to follow for judges from any profession. (TJH)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Note: Paper presented at the Annual Meeting of the American Educational Research Association (New Orleans, LA, April 5-9, 1988).