In the National University of Ireland Maynooth, in-coming first-year mathematics students are given a diagnostic test. The purpose of this test is to measure their basic knowledge and manipulation skills in mathematics. Those who fail this test are deemed at risk of failing their mathematics modules and are enrolled in a supplementary online mathematics proficiency course. In this article, we give a brief overview of the development of this course. We focus on the issue of student engagement with the course and the steps we have taken to address their levels of engagement; in particular, we have monitored student activity in the online course and contacted students whose engagement levels were low. We also present an analysis of the impact of this course on students' overall grades.