We report on research carried out to improve teaching and student engagement in the introductory astronomy course at the University of Cape Town. This course is taken by a diverse range of students, including many from educationally disadvantaged backgrounds. We describe the development of an instrument, the Introductory Astronomy Questionnaire (IAQ), which we administered as pre- and posttests to students enrolled in the course. The instrument comprised a small number of questions which probed three areas of interest: student motivation and expectations, astronomy content, and worldview. Amongst our findings were that learning gains were made in several conceptual areas, and that students appeared to develop a more nuanced view of the nature of astronomy. There was some evidence that the course had a positive impact on students' worldviews, particularly their attitudes towards science. We also identified a promising predictor of course success that could in the future be used to identify students requiring special teaching intervention.