This study was designed to provide multiple sources of evidence of the validity of the Creative Achievement Questionnaire (CAQ) and to clarify the hierarchy of creative achievement using Rasch analyses. A total of 905 Taiwanese participants (345 men and 558 women) completed the CAQ online. The Rasch model was used to assess model-data fit. A differential item functioning (DIF) analysis was conducted to assess the consistency of the ratings provided by males and females. The results revealed that the 10 dimensions of the CAQ showed good model-data fit and supported the two-factor classification (arts and sciences). Additionally, several items from each domain exhibited substantial DIF across genders. Moreover, with the exception of dance, creative writing, architectural design, theater and film, and culinary arts, items in the 10 domains were hierarchically ordered. Suggestions for future research to revise the CAQ were proposed.