The aim of this study is to develop a scoring rubric to assess primary school students' problem posing skills. The rubric including five dimensions namely solvability, reasonability, mathematical structure, context and language was used. The raters scored the students' problem posing skills both with and without the scoring rubric to test the reliability of the rubric. The study used generalizability theory to test the reliability of the scores obtained with and without the use of the rubric. More reliable scoring was obtained using the scoring rubric. The G and phi coefficients rose somewhat after increasing the number of items and raters in both scoring methods. However, increasing the number of items affected these coefficients slightly more than increasing the number of the raters.