College admissions leaders increasingly desire to incorporate non-cognitive factors like grit into admissions decisions. Consequently, we examined the validity of the short grit scale (Grit-S) using data collected on undergraduates attending 38 colleges. Using confirmatory factor analysis (CFA), we found that Grit-S does not possess adequate model fit; however, a modified version of the scale does possess this property. Using multi-group CFA, we also found that Grit-S is relatively invariant across multiple demographic groups. Next, we examined the concurrent validity of grit with students' engagement, perceived gains, time spent studying, and grades. These results confirmed previous research that most of the predictive power of grit is contained in its perseverance of effort, not consistency of interest, dimension.