This article explores the potential of cross-national achievement studies through a critical reflection upon the experience of South Africa's participation in the Trends in International Mathematics and Science Study (TIMSS). There are a growing number of multi-country and international achievement studies. While recognizing the limitations of these studies, this article considers how far these studies can be an effective tool for the generation of in-country information that can usefully inform policy and practice. It is argued that improvements can be made by using a simpler methodology for the analysis and presentation of information; better ways of analysing the influence of domestic dimensions of poverty, inequality and social stratification; and improved strategies for assessing progress over time against national baselines. The article concludes with a set of questions for systems to consider when thinking about participating in international or multi-country achievement studies.