ERIC Number: ED445024
Record Type: Non-Journal
Publication Date: 2000-Apr
Reference Count: N/A
Kohonen Self-Organizing Maps in Validity Maintenance for Automated Scoring of Constructed Response.
Williamson, David M.; Bejar, Isaac I.
As the automated scoring of constructed responses reaches operational status, monitoring the scoring process becomes a primary concern, particularly if automated scoring is intended to operate completely unassisted by humans. Using actual candidate selections from the Architectural Registration Examination (n=326), this study uses Kohonen Self-Organizing Maps (SOM) to build on previous research (D. Williamson, A. Hone, S. Miller, and I. Bejar, 1998) suggesting that classification trees are a useful means of validity maintenance. Classification trees can assist in identifying sources of disagreement between human and automated scoring, identify tendencies for human graders to overlook elementary or complex solutions, and provide significant efficiency in future case selection for human intervention. Since classification trees require a criterion value of score discrepancy between human and automated scores, Kohonen SOM provide an advantage in the ability to classify solutions in similar groups through neural networks without requiring prior human grading. Results suggest that Kohonen SOM could be used to classify solutions prior to human grading and classification tree analyses, thus providing a 43% reduction in the human grading required. However, further analyses are needed to establish whether classification trees would produce similar results with a reduced sample on the basis of Kohonen SOM classifications. (Contains 3 figures, 3 tables, and 14 references.) (Author/SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A