NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ953400
Record Type: Journal
Publication Date: 2012
Pages: 14
Abstractor: As Provided
Reference Count: 11
ISSN: ISSN-0895-7347
Comparison of Human and Machine Scoring of Essays: Differences by Gender, Ethnicity, and Country
Bridgeman, Brent; Trapani, Catherine; Attali, Yigal
Applied Measurement in Education, v25 n1 p27-40 2012
Essay scores generated by machine and by human raters are generally comparable; that is, they can produce scores with similar means and standard deviations, and machine scores generally correlate as highly with human scores as scores from one human correlate with scores from another human. Although human and machine essay scores are highly related on average, this does not eliminate the possibility that machine and human scores may differ significantly for certain gender, ethnic, or country groups. Such differences were explored with essay data from two large-scale high-stakes testing programs: the Test of English as a Foreign Language and the Graduate Record Examination. Human and machine scores were very similar across most subgroups, but there were some notable exceptions. Policies were developed so that any differences between humans and machines would have a minimal impact on final reported scores. (Contains 7 tables and 1 footnote.)
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site:
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Graduate Record Examinations; Test of English as a Foreign Language