Science, Technology, Engineering, and Mathematics (STEM) education has become a key focus in the U.S. government's public education agenda. Many STEM degrees require the successful completion of undergraduate introductory organic chemistry (O-Chem), which is notorious for its difficulty and high attrition rate. Concept Maps (CM) have been used as a tool to improve teaching and learning by providing feedback to teacher and students. Although numerous studies have examined the use of concept maps (CMs) as an assessment tool in science classes, none to date has examined such applications of CMs in O-Chem. Furthermore, studies investigating the validity of CMs in post-secondary science courses are rare. Thus, the present study investigated the validity of CMs as an assessment tool and their diagnostic uses in O-Chem by examining the relationship between CM scores and other key performance measures in O-Chem. Results indicated that CM scores were significantly correlated with problem set scores and final course grade. In addition, a mediation analysis revealed that problem solving scores partially mediated the relationship between problem solving and final course grade, confirming the role CMs are expected to play in O-Chem achievement. Implications for using CMs as diagnostic and formative tools in instruction are discussed. (Contains 3 notes, 4 tables, and 4 figures.)