ERIC Number: EJ1099513
Record Type: Journal
Publication Date: 2016-Jun
Abstractor: As Provided
The Impact of Misspelled Words on Automated Computer Scoring: A Case Study of Scientific Explanations
Ha, Minsu; Nehm, Ross H.
Journal of Science Education and Technology, v25 n3 p358-374 Jun 2016
Automated computerized scoring systems (ACSSs) are being increasingly used to analyze text in many educational settings. Nevertheless, the impact of misspelled words (MSW) on scoring accuracy remains to be investigated in many domains, particularly jargon-rich disciplines such as the life sciences. Empirical studies confirm that MSW are a pervasive feature of human-generated text and that despite improvements, spell-check and auto-replace programs continue to be characterized by significant errors. Our study explored four research questions relating to MSW and text-based computer assessments: (1) Do English language learners (ELLs) produce equivalent magnitudes and types of spelling errors as non-ELLs? (2) To what degree do MSW impact concept-specific computer scoring rules? (3) What impact do MSW have on computer scoring accuracy? and (4) Are MSW more likely to impact false-positive or false-negative feedback to students? We found that although ELLs produced twice as many MSW as non-ELLs, MSW were relatively uncommon in our corpora. The MSW in the corpora were found to be important features of the computer scoring models. Although MSW did not significantly or meaningfully impact computer scoring efficacy across nine different computer scoring models, MSW had a greater impact on the scoring algorithms for naïve ideas than key concepts. Linguistic and concept redundancy in student responses explains the weak connection between MSW and scoring accuracy. Lastly, we found that MSW tend to have a greater impact on false-positive feedback. We discuss the implications of these findings for the development of next-generation science assessments.
Descriptors: Spelling, Case Studies, Computer Uses in Education, Test Scoring Machines, Scoring, Biological Sciences, English Language Learners, Computer Assisted Testing, Educational Technology, Error Patterns, Models, Student Reaction, Feedback (Response), Spelling Instruction, Error Analysis (Language), Science Instruction, Scientific Concepts
Springer. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: firstname.lastname@example.org; Web site: http://www.springerlink.com
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A