ERIC Number: ED454239
Record Type: RIE
Publication Date: 2001-Apr
Identifying Undifferentiating Response Sets and Assessing Their Effects on the Measurement of Items.
Schulz, E. Matthew; Sun, Anji
Undifferentiating response sets, defined as "overuse" of any category of a Likert scale, were identified using a combination of simple criteria, such as whether a single-category response set involved more than four items, and statistical criteria based on D. Andrich's (1978) measurement model for Likert scales (the Rating Scale model). Data were from one section of the American College Testing Program's "Counseling for High Skills" survey for 10 colleges. Total counts across colleges for the 4 response sets were, respectively: 5,254; 4,757; 4,411; and 4,212. Undifferentiating response sets were strongly associated with statistically significant person misfit in Rating Scale model analyses. When persons with undifferentiating response sets were removed from the sample, the reliability of the item measures improved, and the rank order of the items became more internally consistent. It is concluded that applications of measurement theory can be useful in evaluating the quality of survey data. (Contains 4 figures, 9 tables, and 11 references.) (SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Note: Paper presented at the Annual Meeting of the American Educational Research Association (Seattle, WA, April 10-14, 2001).