Recent years have seen a shift in focus from assessment of learning to assessment for learning and the emergence of alternative assessment methods. However, the reliability and validity of these methods as assessment tools are still questionable. In this article, we investigated the predictive validity of measures of the Pathfinder Scaling Algorithm (PSA), a concept mapping assessment utility, using the referent-free and referent-based approaches on programming performance of a group of secondary school students. Results suggest that the predictive validity of both approaches was more or less the same. Among the three similarity measures applied for the referent-based approach, PRX appeared to be the most predictive one whereas PFC and GTD were similar in terms of predictive power. The correlations between the referent-free measure C and the three previously mentioned referent-based measures with the programming performance measures were not as strong as reported in the literature. In the light of these results, we argue that there is a need to reform assessment in programming education. (Contains 2 figures and 5 tables.)