NotesFAQContact Us
Search Tips
ERIC Number: ED454257
Record Type: Non-Journal
Publication Date: 2001-Apr
Pages: 35
Abstractor: N/A
Reference Count: N/A
Quantifying Quality: Technical Issues Surrounding "U.S. News and World Report's" Rankings of U.S. Colleges and Graduate Schools.
Clarke, Marguerite
This paper examines two common criticisms of the method that "U.S. News and World Report" used in its annual rankings of the academic quality of colleges and graduate schools. There are constant changes to the formula that make it impossible to interpret yearly shifts in a school's relative ranking in terms of change in its academic quality and the overly precise nature of the scores used to rank schools. This paper does not judge the overall concept of academic quality rankings and the validity of different indicators; it focuses on the extent to which methodological issues may affect the interpretation of the "U.S. News" rankings. To gauge the changes in ranking formulas, four types of change were identified and tracked over 6 years: (1) changes in the weight assigned to an indicator; (2) the removal of an indicator; (3) the addition of an indicator; and (4) changes in an indicator's definition or methodology. Five graduate discipline rankings and two undergraduate rankings (national university and national liberal arts) were examined. Analyses show that it is generally not possible to interpret year-to-year shifts in a school's rank in terms of its academic quality. It is possible, depending on the ranking, to make cross-year comparisons of a school's relative performance on between one-third and three-quarters of the individual indicators used. Results of the error analyses call into question the use of overall scores to assign schools to individual ranks. Recommendations are made for improving the interpretability and usefulness of the "U.S. News" rankings. Current "U.S. News" College and Graduate School Ranking Methodology is appended. (Contains 15 tables and 15 references.) (SLD)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A