ERIC Number: ED336046
Record Type: RIE
Publication Date: 1991-May
Reference Count: N/A
Lies, Damn Lies, and Statistics Revisited: A Comparison of Three Methods of Representing Change. AIR 1991 Annual Forum Paper.
Pike, Gary R.
Because change is fundamental to education and the measurement of change assesses the quality and effectiveness of postsecondary education, this study examined three methods of measuring change: (1) gain scores; (2) residual scores; and (3) repeated measures. Data for the study was obtained from transcripts of 722 graduating seniors at the University of Tennessee, Knoxville. The gain method involves administering an instrument at the beginning of a program of study and then readministering the instrument at the end of the program. Residual scores are calculated by regressing students' scores at the end of a program of study on their entering scores in order to develop a prediction model. The difference between actual and predicted scores then represents student change. The repeated measures method uses all of the data from the two tests to describe change. Results of the analysis and comparison found that all three methods were marred by similar problems of unreliability. Reliability coefficients and large standard errors of measurement suggested that what is being measured is not true change, but error. However, the repeated measures technique offered the greatest potential because it maintains the original test data, allowing researchers to bring more information to bear in interpreting their findings. One table, one figure and 41 references accompany the text. (JB)
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: N/A
Sponsor: Fund for the Improvement of Postsecondary Education (ED), Washington, DC.
Authoring Institution: N/A
Identifiers: AIR Forum; Gain Scores; Repeated Measures Design; Residual Scores; University of Tennessee Knoxville
Note: Paper presented at the Annual Forum of the Association for Institutional Research (31st, San Francisco, CA, May 26-29, 1991).