ERIC Number: EJ894508
Record Type: Journal
Publication Date: 2010-Sep
Abstractor: As Provided
Marking Essays on Screen: An Investigation into the Reliability of Marking Extended Subjective Texts
Johnson, Martin; Nadas, Rita; Bell, John F.
British Journal of Educational Technology, v41 n5 p814-826 Sep 2010
There is a growing body of research literature that considers how the mode of assessment, either computer-based or paper-based, might affect candidates' performances. Despite this, there is a fairly narrow literature that shifts the focus of attention to those making assessment judgements and which considers issues of assessor consistency when dealing with extended textual answers in different modes. This research project explored whether the mode in which a set of extended essay texts were accessed and read systematically influenced the assessment judgements made about them. During the project, 12 experienced English literature assessors marked two matched samples of 90 essay exam scripts on screen and on paper. A variety of statistical methods were used to compare the reliability of the essay marks given by the assessors across modes. It was found that mode did not present a systematic influence on marking reliability. The analyses also compared examiners' marks with a gold standard mark for each essay and found no shifts in the location of the standard of recognised attainment across modes.
Descriptors: English Literature, Examiners, Evaluation Research, Evaluators, Student Evaluation, Grading, Essays, Reliability, Printed Materials, Interrater Reliability, Essay Tests, Comparative Analysis, Computer Assisted Testing, Educational Technology
Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: email@example.com; Web site: http://www.wiley.com/WileyCDA/
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Authoring Institution: N/A