ERIC Number: EJ1250358
Record Type: Journal
Publication Date: 2020-May
Pages: 17
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0958-3440
EISSN: N/A
Crowdsourcing Evaluation of the Quality of Automatically Generated Questions for Supporting Computer-Assisted Language Teaching
ReCALL, v32 n2 p145-161 May 2020
How can state-of-the-art computational linguistic technology reduce the workload and increase the efficiency of language teachers? To address this question, we combine insights from research in second language acquisition and computational linguistics to automatically generate text-based questions to a given text. The questions are designed to draw the learner's attention to target linguistic forms -- phrasal verbs, in this particular case -- by requiring them to use the forms or their paraphrases in the answer. Such questions help learners create form-meaning connections and are well suited for both practice and testing. We discuss the generation of a novel type of question combining a "wh"- question with a gapped sentence, and report the results of two crowdsourcing evaluation studies investigating how well automatically generated questions compare to those written by a language teacher. The first study compares our system output to gold standard human-written questions via crowdsourcing rating. An equivalence test shows that automatically generated questions are comparable to human-written ones. The second crowdsourcing study investigates two types of questions ("wh"- questions with and without a gapped sentence), their perceived quality, and the responses they elicit. Finally, we discuss the challenges and limitations of creating and evaluating question-generation systems for language learners.
Descriptors: Computational Linguistics, Computer Assisted Testing, Computer Assisted Instruction, Second Language Instruction, Language Teachers, Questioning Techniques, Language Tests
Cambridge University Press. 100 Brook Hill Drive, West Nyack, NY 10994. Tel: 800-872-7423; Tel: 845-353-7500; Fax: 845-353-4141; e-mail: subscriptions_newyork@cambridge.org; Web site: https://journals.cambridge.org
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A