NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1046018
Record Type: Journal
Publication Date: 2014
Pages: 9
Abstractor: As Provided
Reference Count: 15
Combining the Best of Two Standard Setting Methods: The Ordered Item Booklet Angoff
Smith, Russell W.; Davis-Becker, Susan L.; O'Leary, Lisa S.
Journal of Applied Testing Technology, v15 n1 p18-26 2014
This article describes a hybrid standard setting method that combines characteristics of the Angoff (1971) and Bookmark (Mitzel, Lewis, Patz & Green, 2001) methods. The proposed approach utilizes strengths of each method while addressing weaknesses. An ordered item booklet, with items sorted based on item difficulty, is used in combination with individual item ratings rather than a bookmark placement. The ordered item booklet attempts to make individual item ratings more palatable by providing some amount of information about item difficulty to the Subject Matter Experts (SMEs) participating in the standard setting. Allowing for individual item ratings attempts to addresses the issue of SMEs disagreeing with the order of items in the booklet in a Bookmark standard setting. This article illustrates the method as it was applied to a healthcare examination used for state licensure, comparing it with a more traditional Angoff approach. Results suggest that the proposed method has promise for future use pending further exploration.
Association of Test Publishers. 601 Pennsylvania Avenue NW, South Building Suite 900, Washington DC 20004. Tel: 866-240-7909; Fax: 717-755-8962; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A