NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ982601
Record Type: Journal
Publication Date: 2012
Pages: 26
Abstractor: As Provided
Reference Count: 29
ISSN: ISSN-0735-6331
On the Reliability and Validity of Human and LSA-Based Evaluations of Complex Student-Authored Texts
Seifried, Eva; Lenhard, Wolfgang; Baier, Herbert; Spinath, Birgit
Journal of Educational Computing Research, v47 n1 p67-92 2012
This study investigates the potential of a software tool based on Latent Semantic Analysis (LSA; Landauer, McNamara, Dennis, & Kintsch, 2007) to automatically evaluate complex German texts. A sample of N = 94 German university students provided written answers to questions that involved a high amount of analytical reasoning and evaluation. LSA-based evaluations were compared to evaluations of six human graders. Results showed that LSA-based evaluations agreed as well with human graders as human graders agreed with each other. Agreement of human graders and LSA-based evaluations did not differ depending on the standard of comparison used by LSA, that is, whether scoring was based on one single text ("gold standard") or on a sample of previously scored assignments ("nearest neighbors"). Moreover, LSA-based evaluations of students' assignments predicted students' results in a final exam. These results show that LSA can support university teachers in giving feedback and grading complex German texts. (Contains 4 tables and 1 figure.)
Baywood Publishing Company, Inc. 26 Austin Avenue, P.O. Box 337, Amityville, NY 11701. Tel: 800-638-7819; Tel: 631-691-1270; Fax: 631-691-1770; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Germany