This paper discusses the National Assessment Program for Literacy and Numeracy (NAPLAN) performance of Years Five, Seven and Nine students in standardised tests prior and post the implementation of a mobile learning initiative in a Western Australian school for boys. The school sees the use of ICT as important in enhancing its potential to deliver optimal educational outcomes. However, the School is also cognisant of the shared concern of teachers and parents in the school community about an over-reliance on mobile devices for learning, to the detriment of students' accomplishments in literacy and numeracy. The paper examines NAPLAN results from standardised test scores prior and post the mobile learning initiative at the School and in comparison to national data. Literacy and numeracy results were analysed between two periods: prior to 2010 in which the mobile learning initiative was not implemented at the School and between 2010 and 2012, in which the mobile learning initiative was implemented in Years Five, Seven and Nine. It is argued that the implementation of mobile learning has had a minimal effect on student performance as gauged by standardised testing.