In this study, we developed a Reading for Meaning special education teacher observation rubric that detailed the elements of evidence-based comprehension instruction and tested its psychometric properties using many-facet Rasch measurement. We collected video observations of classroom instruction from 10 special education teachers across 3 states during the 2015-2016 school year. External raters (n = 4) trained to observe and evaluate instruction using the rubric assigned scores of implemented, partially implemented, or not implemented for each of the items. Analyses showed that the item, teacher, lesson, and rater facets achieved high psychometric quality. Teacher performance was consistent with what the literature has reported. We discuss implications for research and practice.