First-year engineering (FYE) students are routinely screened for spatial ability, with the goals of predicting retention in the major and identifying those who need supplementary spatial instruction. Psychometric tests used for such screenings are often domain-general measures that lack diagnostic information to inform remedial instruction. A new approach to spatial screening is to use measures that assess performance on authentic engineering tasks while accounting for the cognitive processes that underlie spatial thinking. We tested the utility of a relatively new spatial visualization test (the Santa Barbara Solids Test; SBST) to characterize individual differences in performance among FYE students with low mental rotation scores. The internal reliability and predictive validity of the SBST were previously demonstrated in sample populations with average spatial skill. One hundred and forty-one FYE students with low mental rotation scores completed the SBST and an engineering drawing task before instruction. We investigated the internal reliability of the SBST, patterns of performance and the validity of the test to predict performance on the drawing task. Through item analysis, we deleted problems that contributed to low internal reliability. Performance means were normally distributed. There were small significant positive correlations between the drawing task and SBST total score and subscales. The SBST shows promise for diagnosing difficulties and strategies demonstrated by students who are challenged by spatial visualization. We suggest applications of the SBST to support remedial spatial instruction.