ERIC Number: ED520485
Record Type: Non-Journal
Publication Date: 2010
Reference Count: 53
Do College Student Surveys Have Any Validity?
Porter, Stephen R.
Association for Institutional Research (NJ1), Paper presented at the Annual Forum of the Association for Institutional Research (50th, Chicago, IL, May 29-Jun 2, 2010)
Within the field of higher education, the majority of quantitative research focuses on college students. Given the limitations of institutional databases, surveys of college students have become one of the largest and most frequently used data sources. In addition, surveys of college students play an increasingly important role in evaluating college and university programs and policies. As such, having valid and reliable data about students is vital for both practitioners and scholars. Yet if survey questions are not measuring what individuals think they are, then their knowledge of college students will be flawed. In this paper, the author argues that the typical college student survey question has minimal validity, and that the field requires an ambitious research program to reestablish the foundation of quantitative research on students. The surveys lack validity because a) they assume that college students can easily report information about their behaviors and attitudes, when the standard model of human cognition and survey response clearly suggests they cannot, b) existing research using college students suggests they have problems correctly answering even simple questions about factual information, and c) much of the evidence that higher education scholars cite as evidence of validity and reliability actually demonstrates the opposite. The author chooses the National Survey of Student Engagement (NSSE) for his critical examination of college student survey validity for several reasons. First, it is one of the most prominent surveys of student behavior and attitudes, and is widely used by researchers studying students, as well as institutions interested in assessment. Second, the NSSE survey serves as a model for surveys designed by other researchers and institutional assessment staff, precisely because of its prominence. Given its wide use by both practitioners and scholars, it is vital that individuals understand whether the NSSE can be considered a valid instrument. Finally, unlike many other college student surveys, NSSE staff and researchers using the NSSE (henceforth collectively referred to as NSSE researchers) have gone to significant efforts to validate the survey through a variety of studies. (Contains 4 tables, 1 figure and 3 footnotes.)
Descriptors: Higher Education, College Students, Student Behavior, Student Evaluation, Student Surveys, Test Validity, Evaluation Methods, Statistical Analysis, Evidence, Comprehension
Association for Institutional Research. 1435 East Piedmont Drive Suite 211, Tallahassee, FL 32308. Tel: 850-385-4155; Fax: 850-383-5180; e-mail: firstname.lastname@example.org; Web site: http://www.airweb.org
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: Higher Education
Authoring Institution: Association for Institutional Research
Identifiers - Assessments and Surveys: National Survey of Student Engagement