NotesFAQContact Us
Collection
Advanced
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: ED592713
Record Type: Non-Journal
Publication Date: 2016
Pages: 6
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
How Good Is Popularity? Summary Grading in Crowdsourcing
Li, Haiying; Cai, Zhiqiang; Graesser, Arthur
International Educational Data Mining Society, Paper presented at the International Conference on Educational Data Mining (EDM) (9th, Raleigh, NC, Jun 29-Jul 2, 2016)
In this paper, we applied the crowdsourcing approach to develop an automated popularity summary scoring, called wild summaries. In contrast, the golden standard summaries generated by one or more experts are called expert summaries. The innovation of our study is to compute LSA (Latent Semantic Analysis) similarities between target summary and wild summaries rather than expert summaries. We called this method CLSAS, i.e., crowdsourcing-based LSA similarity. We evaluated CLSAS by comparing it with other approaches, Coh-Metrix language and discourse features and LIWC psychometric word measures. Results showed that CLSAS alone could explain 19% of human summary score, which was equivalent to the variance explained by dozens of language and discourse features and/or the word features. Results also showed that adding language and/or word features to CLSAS increased small additional correlations. Findings imply that crowdsourcing-based LSA similarity approach is a promising method for automated summary assessment. [For the full proceedings, see ED592609.]
International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: http://www.educationaldatamining.org
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED); National Science Foundation (NSF)
Authoring Institution: N/A
IES Funded: Yes
Grant or Contract Numbers: R305A080594; R305G020018; R305C120001; R305A130030; 0325428; 633918; 0834847; 0918409; 1108845