There is increasing pressure for institutions of higher education in the United States to objectively document student learning outcomes. Criticism of higher education is mounting, with the result being that education institutions need to be more accountable for student learning on their campuses. As a result, there is increased interest in the viability and use of standardized tests to assess student academic achievement in higher education. The use of standardized exams is often referred to as "direct assessment" of student learning and considered a powerful tool for assessing learning outcomes. However, such exams often have a shortcoming that is not always recognized or acknowledged. This study aims to investigate test performance differences among undergraduate students under low- and high-stakes test conditions. The authors used data from a college-level test of general education (College Base) that is employed in both high-stakes and low-stakes contexts. The data were examined by using multivariate analysis of covariance (MANCOVA). The results indicate that there is a significant difference in test performance between high- and low-stakes examinees on a general education exam, though the effect sizes were small. The study also found no meaningful differences in the reliability of the data for either testing situation, indicating that these data are providing reliable measures of student achievement in both testing situations. Implications of this study are discussed. (Contains 3 tables.)