Obtaining quick and reliable evidence regarding the proficiency of learners is a perennial issue in second language (L2) learning research. In this study, we examined naturalistic utterance recall as a measure of L2 learning proficiency that can be easily extracted from videos and automatically scored using the video's captions. In our recall task, learners listen to audio clips and write down as much of the utterance as they can remember. We evaluated this naturalistic recall task with a sample of English native speakers who are learning Spanish at beginner to advanced levels, as well as Spanish native speakers. The results suggest that our recall measure is a better predictor of a learner's ability to translate heard sentences than a shortened version of a standardized listening multiple-choice comprehension test. Our findings suggest naturalistic utterance recall can offer an accurate and efficient method for predicting foreign language proficiency.