In the current scenario of expansion of neo-liberal and market-oriented policies, large-scale assessment has become an increasingly pervasive technology of power. Given their consequences, and the high stakes derived from them, assessment systems put pressure on schools to an extent where their practices and internal policies adapt to the logic of these mechanisms. The behaviours that emerge from this adaptation are seen as 'gaming the system' and are often portrayed as the responsibility of test users. However, the article aims at pushing the question around the consequences of metrics further, in terms of what these strategic gaming behaviours are adapting to and to what extent predominant metrics are accomplishing their assumptions of social justice. Through theoretically discussing the concept of social justice in connection with educational assessment, the paper problematises whether current metrics and their consequences, on the one hand, are able to respond to their underlying principles of equality of opportunity, which are characteristic of justice as distribution, and, on the other hand, highlights the insufficiency of these principles to respond to current developments of the concept of social justice with regard to the dimensions of recognition and participation, with considerations for their potential inclusion in future assessment systems.