ERIC Number: EJ1071208
Record Type: Journal
Publication Date: 2015
Abstractor: As Provided
Reference Count: 39
Evaluating the Operational Feasibility of Using Subsets of Items to Recommend Minimal Competency Cut Scores
Kannan, Priya; Sgammato, Adrienne; Tannenbaum, Richard J.
Applied Measurement in Education, v28 n4 p292-307 2015
Establishing cut scores using the Angoff method requires panelists to evaluate every item on a test and make a probability judgment. This can be time-consuming when there are large numbers of items on the test. Previous research using resampling studies suggest that it is possible to recommend stable Angoff-based cut score estimates using a content-stratified subset of ~45 items. Recommendations from earlier work were directly applied in this study in two operational standard-setting meetings. Angoff cut scores from two panels of raters were collected at each study, wherein one panel established the cut score based on the entire test, and another comparable panel first used a proportionally stratified subset of 45 items, and subsequently used the entire test in recommending the cut scores. The cut scores recommended for the subset of items were compared to the cut scores recommended based on the entire test for the same panel, and a comparable independent panel. Results from both studies suggest that cut scores recommended using a subset of items are comparable (i.e., within one standard error) to the cut score estimates from the full test.
Descriptors: Cutting Scores, Test Items, Standard Setting (Scoring), Feasibility Studies, Middle School Teachers, Secondary School Teachers, Licensing Examinations (Professions), Minimum Competency Testing
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: Middle Schools; Secondary Education; Junior High Schools
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Praxis Series