Performance assessments attempt to provide a practical and authentic demonstration of students' learning. Despite growing investments in performance assessments by states, as well as researchers' theorized value of this type of assessment, the field has not developed a measure of assessment literacy specific to performance assessments that has sufficient psychometric evidence to support it. This study begins important research on developing a quantitative measure that can be used by educational practitioners to self-evaluate their own performance assessment literacy (PAL). Using the Quality Performance Assessment (QPA) framework from the Center for Collaborative Education as a foundation, this study explores and confirms the dimensionality of a 27-item survey instrument that assesses educational practitioners' perceptions of their PAL using exploratory and confirmatory factor analysis. Our findings provide evidence that the instrument captures five reliable dimensions of PAL: valid design, reliable scoring, data analysis, fair assessment, and student voice and choice.