The fact that Turkey's achievement in large-scale international examinations is not at the expected level is associated with such factors as students, teachers, schools, curricula and assessment approaches. The literature links student-related causes mainly to their lack of higher-order thinking skills; and teacher-related causes to incompetency in writing items to measure these skills, and not participating in professional development (PD) activities aimed at measurement of the skills tested in these exams. For these reasons, the present study designed, implemented and evaluated a PD program to improve secondary math teachers' skills of writing higher-order items in compliance with Cognitive Diagnostic Models. The study was conducted with Guskey's five levels of evaluating teachers' PD in the holistic single-case design. The program was attended by 100 participants from 20 different cities of Turkey. During the study, both quantitative and qualitative data concerning the periods before, during and after the program implementations were obtained through pre and post evaluation questionnaires and worksheets. The results indicated that the PD program was satisfactory; however, it needed improvement in terms of the total time of the program, time allocated for activities and monitoring and supporting teachers' work at their institutions of service. In addition, it was revealed that the context should not be structured in writing higher-order items of mathematics, which involves multiple qualities and mathematical competence. [This paper was presented at the International Conference on Curriculum and Instruction (5th, Mugla, Turkey, October 26-28, 2017.]