Closed captioning of instructional videos is a topic that has not seen much discussion despite its importance for hearing-impaired students and recent legal ramifications if videos are not appropriately captioned. In particular, it is unclear what best practice in captioning videos should be to benefit all learners in disciplines such as mathematics with a reliance on the development of visual explanation while providing audio narration. In this paper, we report on a study undertaken at an Australian university, to investigate the perceived level of usefulness of captions and their automatic translations in a mathematics course. We discovered that students broadly agreed that captions are a useful learning feature: to allow flexibility of where and when a video is watched, but also to help understand speaker accents, and clarify explanations that are difficult to hear in the recording. Due to the high levels of use and perceived educational benefits of closed captions in online video but limited literature, there is a significant need for new research in this area. An urgent discussion is needed to explore how students engage with closed captions, how they may support learning, and to investigate implications on instructional design of mathematical videos.