ERIC Number: EJ1151450
Record Type: Journal
Publication Date: 2017-Sep
Abstractor: As Provided
Measuring Longitudinal Gains in Student Learning: A Comparison of Rasch Scoring and Summative Scoring Approaches
Zhao, Yue; Huen, Jenny M. Y.; Chan, Y. W.
Research in Higher Education, v58 n6 p605-616 Sep 2017
This study pioneers a Rasch scoring approach and compares it to a conventional summative approach for measuring longitudinal gains in student learning. In this methodological note, our proposed methodology is demonstrated using an example of rating scales in a student survey as part of a higher education outcome assessment. Such assessments have become increasingly important worldwide for purposes of institutional accreditation and accountability to stakeholders. Data were collected from a longitudinal study by tracking self-reported learning outcomes of individual students in the same cohort who completed the student learning experience questionnaire (SLEQ) in their first and final years. Rasch model was employed for item calibration and latent trait estimation, together with a scaling procedure of concurrent calibration incorporating a randomly equivalent group design and a single group design to measure the gains in self-reported learning outcomes as yielded by repeated measures. The extent to which Rasch scoring compared to the conventional summative scoring method in its sensitivity to change was quantified by a statistical index namely relative performance (RP). Findings indicated greater ability to capture learning outcomes gains from Rasch scoring over the conventional summative scoring method, with RP values ranging from 3 to 17% in the cognitive, social, and value domains of the SLEQ. The Rasch scoring approach and the scaling procedure presented in the study can be readily generalised to studies using rating scales to measure change in student learning in the higher education context. The methodological innovations and contributions of this study are discussed.
Descriptors: Item Response Theory, Scoring, Comparative Analysis, Summative Evaluation, Longitudinal Studies, Achievement Gains, Learning, Rating Scales, Student Surveys, College Outcomes Assessment, College Students, Learning Experience, Questionnaires, Change
Springer. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: email@example.com; Web site: http://www.springerlink.com
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education
Authoring Institution: N/A
Grant or Contract Numbers: N/A