ERIC Number: ED483407
Record Type: Non-Journal
Publication Date: 2004-Nov
Application of Generalizability Theory to Concept-Map Assessment Research. CSE Report 640
Yin, Yue; Shavelson, Richard J.
Center for Research on Evaluation Standards and Student Testing CRESST
In the first part of this paper we discuss the feasibility of using Generalizability (G) Theory to examine the dependability of concept map assessments and to design a concept map assessment for a particular practical application. In the second part, we apply G theory to compare the technical qualities of two frequently used mapping techniques: construct-a-map with created linking phrases (C) and construct-a-map with selected linking phrases (S). We explore some measurement facets that influence concept-map scores and estimate how to optimize different concept mapping techniques by varying the conditions for different facets. We found that C and S were not technically equivalent. The G coefficients for S were larger than those for C. Furthermore, a D study showed that fewer items (propositions) would be needed for S than C to reach desired level of G coefficients if only one occasion could be afforded. Therefore, S might be a better candidate than C in the large-scale summative assessment, while C would be preferred as a formative assessment in classroom.
Center for the Study of Evaluation (CSE)/National Center for Research on Evaluation, Standards, and Student Testing (CRESST), 300 Charles E. Young Drive North, GSE&IS Bldg., 3rd Flr./Mailbox 951522, Los Angeles, CA 90095-1522. Tel: 310-206-1532.
Publication Type: Reports - Evaluative
Education Level: N/A
Sponsor: Institute of Education Sciences (ED), Washington, DC.
Authoring Institution: California Univ., Los Angeles. Center for the Study of Evaluation.