NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ893148
Record Type: Journal
Publication Date: 2010-Sep
Pages: 25
Abstractor: As Provided
Reference Count: 25
ISSN: ISSN-0267-1522
Evaluating the Rank-Ordering Method for Standard Maintaining
Bramley, Tom; Gill, Tim
Research Papers in Education, v25 n3 p293-317 Sep 2010
The rank-ordering method for standard maintaining was designed for the purpose of mapping a known cut-score (e.g. a grade boundary mark) on one test to an equivalent point on the test score scale of another test, using holistic expert judgements about the quality of exemplars of examinees' work (scripts). It is a novel application of an old technique (Thurstone's paired comparison method for scaling psychological stimuli), and one that can be applied when the more familiar methods of statistical equating or item banking are not possible. How should a method like this be evaluated? If the correct mapping were known, then the outcome of a rank-ordering exercise could be compared against that. However, in the contexts for which the method was designed, there is no "right answer". This paper presents an evaluation of the rank-ordering method in terms of its rationale, its psychological validity and the stability of the outcome when various factors incidental to the method are varied (e.g. the number of judges, the number of scripts to be ranked, methods of data modelling and analysis). (Contains 5 figures, 6 tables and 7 notes.)
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site:
Publication Type: Journal Articles; Reports - Evaluative
Education Level: Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: United Kingdom