NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: EJ783935
Record Type: Journal
Publication Date: 2006
Pages: 6
Abstractor: ERIC
Reference Count: 6
ISSN: ISSN-0734-6670
Assessing the Reliability of Ratings Used in Undergraduate Admission Decisions
Kretchmar, Jennifer
Journal of College Admission, n192 p10-15 Sum 2006
Many colleges and universities receive thousands of applications for freshman admission every year. To facilitate the process of evaluating each and every applicant in a relatively short amount of time, schools often devise quantitative ratings scales to summarize student characteristics. The ratings give readers a shorthand way to communicate the qualities of each student, and sometimes play a critical role in determining whether or not a student is offered admission. Given the significance of the decision for both the student and the institution to which he or she is applying, admission officers should have a vested interest in the reliability of their ratings scales and should be asking if students would receive the same ratings if their applications were reviewed by different readers. Ironically, even as admission officers evaluate applicants according to strict standards, they sometimes fail to evaluate their own practices with similar precision. For an accurate rating, admission professionals must determine different sources of variations in scores. Generalizability theory allows for this consideration and, therefore, has several advantages over more traditional measures of reliability. The purpose of this research was to use generalizability theory to investigate the reliability of six ratings scores used when evaluating applicants for admission to a university's undergraduate program. Reliability estimates were generated for these scales for both relative and absolute decisions--for the purpose of rating an individual applicant relative to all other applicants, and for the purpose of making a decision to offer or decline admission. (Contains 2 tables.)
National Association for College Admission Counseling. 1631 Prince Street, Alexandria, VA 22314-2818. Tel: 800-822-6285; Tel: 703-836-2222; Fax: 703-836-8015; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A