ERIC Number: EJ1215051
Record Type: Journal
Publication Date: 2019-May
Pages: 41
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1560-4292
EISSN: N/A
Available Date: N/A
Automated Analysis of Reflection in Writing: Validating Machine Learning Approaches
International Journal of Artificial Intelligence in Education, v29 n2 p217-257 May 2019
Reflective writing is an important educational practice to train reflective thinking. Currently, researchers must manually analyze these writings, limiting practice and research because the analysis is time and resource consuming. This study evaluates whether machine learning can be used to automate this manual analysis. The study investigates eight categories that are often used in models to assess reflective writing, and the evaluation is based on 76 student essays (5080 sentences) that are largely from third- and second-year health, business, and engineering students. To test the automated analysis of reflection in writings, machine learning models were built based on a random sample of 80% of the sentences. These models were then tested on the remaining 20% of the sentences. Overall, the standardized evaluation shows that five out of eight categories can be detected automatically with substantial or almost perfect reliability, while the other three categories can be detected with moderate reliability (Cohen's k ranges between 0.53 and 0.85). The accuracies of the automated analysis were on average 10% lower than the accuracies of the manual analysis. These findings enable reflection analytics that is immediate and scalable.
Descriptors: Reflection, Writing (Composition), Writing Evaluation, Automation, Artificial Intelligence, Essays, College Students, Reliability, Validity
Springer. Available from: Springer Nature. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A

Peer reviewed
Direct link
