ERIC Number: EJ1112577
Record Type: Journal
Publication Date: 2016
Abstractor: As Provided
Integrating the Analysis of Mental Operations into Multilevel Models to Validate an Assessment of Higher Education Students' Competency in Business and Economics
Brückner, Sebastian; Pellegrino, James W.
Journal of Educational Measurement, v53 n3 p293-312 Fall 2016
The Standards for Educational and Psychological Testing indicate that validation of assessments should include analyses of participants' response processes. However, such analyses typically are conducted only to supplement quantitative field studies with qualitative data, and seldom are such data connected to quantitative data on student or item performance. This paper presents an example of how data from an analysis of mental operations collected using a sociocognitive approach can be quantitatively integrated with other data on student and item performance to validate in part an assessment of higher education students' competency in business and economics. Evidence of forward reasoning and paraphrasing as mental operations is obtained using the think-aloud method. As part of the validity argument and to enhance credibility of the findings, the generalized linear models are expressed as multilevel models in which the analyses of response processes are aligned with quantitative findings from large-scale field studies.
Descriptors: Hierarchical Linear Modeling, Test Validity, Statistical Analysis, College Students, Business, Economics, Achievement Tests, Protocol Analysis, Cognitive Processes
Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: email@example.com; Web site: http://www.wiley.com/WileyCDA
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Authoring Institution: N/A
Grant or Contract Numbers: N/A