NotesFAQContact Us
Collection
Advanced
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ797382
Record Type: Journal
Publication Date: 2008-Feb
Pages: 22
Abstractor: As Provided
Reference Count: 24
ISBN: N/A
ISSN: N/A
Matching the Judgmental Task with Standard Setting Panelist Expertise: The Item-Descriptor (ID) Matching Method
Ferrara, Steve; Perie, Marianne; Johnson, Eugene
Journal of Applied Testing Technology, v9 n1 p1-22 Feb 2008
Psychometricians continue to introduce new approaches to setting cut scores for educational assessments in an attempt to improve on current methods. In this paper we describe the Item-Descriptor (ID) Matching method, a method based on IRT item mapping. In ID Matching, test content area experts match items (i.e., their judgments about the knowledge and skills required to respond to an item) to the knowledge and skills described in performance level descriptors that are used for reporting test results. We argue that the cognitive-judgmental task of matching item response requirements to performance level descriptors is aligned closely with the experience and expertise of standard setting panelists, who are typically classroom teachers and other content area experts. Unlike other popular standard setting methods, ID Matching does not require panelists to make error-prone probability judgments, predict student performance, or imagine examinees who are just barely in a performance level. We describe applications of ID Matching in two educational testing programs and provide evidence of the effectiveness of this method. The entire process is described in the first section of the paper. Subsequent sections describe applications of ID Matching for two operational testing programs. (Contains 4 figures, 4 tables and 8 footnotes.)
Association of Test Publishers. 601 Pennsylvania Avenue NW, South Building, Suite 900, Washington, DC 20004. Tel: 866-240-7909; Fax: 717-755-8962; e-mail: atp@att.net; Web site: http://www.testpublishers.org
Publication Type: Journal Articles; Opinion Papers; Reports - Descriptive
Education Level: Adult Education; Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A