NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ983097
Record Type: Journal
Publication Date: 2012
Pages: 20
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1530-5058
EISSN: N/A
Comparison of "E-Rater"[R] Automated Essay Scoring Model Calibration Methods Based on Distributional Targets
Zhang, Mo; Williamson, David M.; Breyer, F. Jay; Trapani, Catherine
International Journal of Testing, v12 n4 p345-364 2012
This article describes two separate, related studies that provide insight into the effectiveness of "e-rater" score calibration methods based on different distributional targets. In the first study, we developed and evaluated a new type of "e-rater" scoring model that was cost-effective and applicable under conditions of absent human rating and small candidate volume. This new model type, called the Scale Midpoint Model, outperformed an existing "e-rater" scoring model that is often adopted by certain "e-rater" system users without modification. In the second study, we examined the impact of three distributional score calibration approaches on existing models' performance. These approaches included percentile calibrations on "e-rater" scores in accordance with a human rating distribution, normal distribution, and uniform distribution. Results indicated that these score calibration approaches did not have overall positive effects on the performance of existing "e-rater" scoring models. (Contains 7 tables and 1 figure.)
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A