The purpose of this study was to explore the reliability of measures of both individual and group creative work using the consensual assessment technique (CAT). CAT was used to measure individual and group creativity among a population of pre-service music teachers enrolled in a secondary general music class (n = 23) and was evaluated from multiple perspectives for reliability. Consistency was calculated using Cronbach's alpha. Judges were found to be highly consistent for individual creativity (a = 0.90), individual craftsmanship (a = 0.87), group creativity (a = 0.86) and group craftsmanship (a = 81). Judges were much less consistent with their ratings of aesthetic sensitivity for individual compositions (a = 0.67) or group performances (a = 0.69). Absolute agreement was calculated by using intraclass correlation coefficient (ICC). Judges were found to be highly in agreement for individual creativity (a = 0.79), individual craftsmanship (a = 0.83), group creativity (a = 0.87) and group craftsmanship (a = 83). Judges were much less in agreement with their ratings of aesthetic sensitivity for individual compositions (a = 0.57) or group performances (a = 0.71). Judges ratings for individual creativity were consistent over time, as evidenced by test-retest reliabilities of 0.89 (creativity), 0.83 (craftsmanship) and 0.79 (aesthetic sensibility). Results indicate, in agreement with prior research, that CAT is a reliable measure of creativity. The researchers introduce the idea that absolute agreement might be a worthwhile construct to explore in future work in the measurement of creativity in music education.