This article reports an original investigation into school performance measures and the multilevel nature of pupil achievement data in the Chilean school system using a sample of 177,461 students, nested within 7146 classrooms, 2283 secondary schools and 313 municipalities. The data-set comprised Year 10 students' 2006 SIMCE test's results in two subject outcomes (language and maths) matched to their prior attainment in grade 8 and family characteristics. The analyses showed the lack of precision of two-level models to draw conclusions regarding the effectiveness of Chilean secondary schools as well as the extent to which different pupil intake, background and context features of Chilean secondary schools influence students' performance. The results show substantial and statistically significant municipal, school and classroom effects in Chile, and how the estimation of school effects changes according to the explanatory variables controlled for in the analysis and the outcome analysed. These results are compared with similar studies carried out in Latin America, as well as in other countries (England and China), in order to situate the findings in the broader knowledge base of Educational Assessment and Effectiveness Research.