ERIC Number: EJ1180133
Record Type: Journal
Publication Date: 2018
Pages: 7
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-1755-1382
EISSN: N/A
Five Go Marking an Exam Question: The Use of Adaptive Comparative Judgement to Manage Subjective Bias
Barber, Jill
Practitioner Research in Higher Education, v11 n1 p94-100 2018
Adaptive Comparative Judgement (ACJ) is an alternative to conventional marking in which the assessor (judge) merely compares two answers and chooses a winner. (Scripts are typically uploaded to the CompareAssess interface as pdf files and are presented side-by-side.) Repeated comparisons and application of the sorting algorithm leads to scripts sorted in order of merit. Boundaries are determined by separate review of scripts. A small pilot of ACJ in the fourth year of the Manchester Pharmacy programme is described. Twelve judges used ACJ to mark 64 scripts previously marked conventionally. 50 students peer-marked their own mock examination question using ACJ. Peer-marking was successful with students learning from the process, and delivering both marks and feedback within two weeks. There was very good consistency among the students acting as judges, and accuracy (as defined by Pollitt, 2012) of 0.94. Staff were similarly consistent, but the agreement with marks obtained by conventional marking was disappointing. While some discrepancies could be attributed to conventional marking failing under the stress of marking during teaching term, the worst discrepancies appeared to originate from inadequate judging criteria. We conclude that ACJ is a very promising method, especially for peer assessment, but that judging criteria require very careful consideration.
Descriptors: Student Evaluation, Alternative Assessment, Scoring, Test Bias, Judges, Value Judgment, Peer Evaluation, Summative Evaluation, Pilot Projects, Feedback (Response), Evaluation Criteria, College Students, Pharmaceutical Education, Accuracy, Case Studies, Foreign Countries
University of Cumbria. Fusehill Street, Carlisle, Cumbria, CA1 2HH, United Kingdom. Tel: +44-1228-616338; e-mail: riple@cumbria.ac.uk; Web site: http://ojs.cumbria.ac.uk/index.php/prhe
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: United Kingdom (Manchester)
Grant or Contract Numbers: N/A