The aim of this study is to evaluate a continuing professional development (CPD) program designed to improve teachers' professional competences about the nature of science (NOS) by using a new evaluation perspective. The researchers followed a year long CPD about NOS with the voluntary attendance of 18 middle school science teachers and their students. In a collaborative and reflective environment, teachers were introduced to various NOS aspects and ways to use explicit instruction and formative assessment in their NOS teaching. In addition, teachers received teaching activities and materials to be implemented in their classrooms for one year. The data were collected and evaluated based on the "five level (learning, beliefs, transfer, results, and reaction) evaluation model". The findings demonstrate that the CPD program about NOS effectively improved teachers' views about NOS, beliefs about teaching and learning the NOS, classroom practices about NOS and also their students' views about NOS. This study is the first to use formative assessment and discourse analysis in a professional development program for in-service teacher education, and also first to evaluate teachers' views, beliefs, practices and their students' views about NOS all together. The findings are thought to motivate researchers to consider multiple level evaluations of future professional development programs.