ERIC Number: EJ1046018
Record Type: Journal
Publication Date: 2014
Abstractor: As Provided
Reference Count: 15
Combining the Best of Two Standard Setting Methods: The Ordered Item Booklet Angoff
Smith, Russell W.; Davis-Becker, Susan L.; O'Leary, Lisa S.
Journal of Applied Testing Technology, v15 n1 p18-26 2014
This article describes a hybrid standard setting method that combines characteristics of the Angoff (1971) and Bookmark (Mitzel, Lewis, Patz & Green, 2001) methods. The proposed approach utilizes strengths of each method while addressing weaknesses. An ordered item booklet, with items sorted based on item difficulty, is used in combination with individual item ratings rather than a bookmark placement. The ordered item booklet attempts to make individual item ratings more palatable by providing some amount of information about item difficulty to the Subject Matter Experts (SMEs) participating in the standard setting. Allowing for individual item ratings attempts to addresses the issue of SMEs disagreeing with the order of items in the booklet in a Bookmark standard setting. This article illustrates the method as it was applied to a healthcare examination used for state licensure, comparing it with a more traditional Angoff approach. Results suggest that the proposed method has promise for future use pending further exploration.
Descriptors: Standard Setting, Difficulty Level, Test Items, Rating Scales, Health Education, Health Services, Licensing Examinations (Professions), State Standards, Evaluation Methods, Methods Research, State Licensing Boards, Test Construction, Convergent Thinking, Test Validity, Test Reliability, Program Descriptions, Group Dynamics
Association of Test Publishers. 601 Pennsylvania Avenue NW, South Building Suite 900, Washington DC 20004. Tel: 866-240-7909; Fax: 717-755-8962; e-mail: firstname.lastname@example.org; Web site: http://www.testpublishers.org
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Authoring Institution: N/A