International large-scale assessments and comparisons (ILSAs) in education have become significant policy phenomena. How a country fares in these assessments has come to signify not only how a nation's education system is performing, but also its future prospects in a global economic "race". These assessments provoke passionate arguments at specialist conferences and in scholarly journals and they are just as passionately debated in the media. Within academe, ILSAs are researched by sociologists and psychometricians, policy experts and statisticians. This multidisciplinary, multi-voice discussion has not always served to highlight the complexity of the issues involved. Instead, discussions across various groups of actors have often led to a polarisation of views and a hardening of stances. Large-scale comparisons have deeply divided academic opinion with regard to their validity, usefulness and use. The divergence in ontological commitments, methodologies and paradigms of research makes discussions among one set of scholars almost incomprehensible to another. New theories, concepts and vocabularies are urgently required to engage productively with this important phenomenon. Borrowing concepts from Science and Technology Studies (STS) and the history and sociology of numbers, I argue that understanding such comparative exercises as socio-technical assemblages would move the critique of large-scale comparisons in education in more productive directions.