Standardized assessment of content knowledge for STEM (Science, Technology, Engineering, Mathematics) topics is pervasive in both K-12 and postsecondary institutions. Yet, most instruments are developed with little to no validation for students with learning disability (LD), attention deficit hyperactivity disorder (ADHD), and autism spectrum disorder (ASD). In this study, we evaluate the effectiveness of new test instruments designed to assess conceptual understanding of statistics content and the extent to which construct-irrelevant factors, such as language processing, influence the performance of students with disabilities. Generalized linear mixed-effects modeling was used to identify the factors that were predictive of student performance. Results indicate that the average sentence length in word problems was uniquely predictive of student performance on emerging assessments of conceptual understanding. The results provide new evidence of the barriers facing students with disabilities on emerging tests of conceptual knowledge. A general framework for measuring the impact of these barriers and the effectiveness of accommodations is discussed.