In instructional sensitivity research, it is important to evaluate the validity argument about the extent to which student performance on the assessment can be used to infer differences in instructional experiences. This study examines whether three different measures of mathematics instruction consistently identify mathematics assessments as being sensitive to instruction. Mixed findings across fourth-grade (n = 8,298) and fifth-grade (n = 9,336) students and their teachers across three school districts raise questions as to whether different ways of measuring instruction provide similar inferences about the instructional sensitivity of assessments. This raises validity concerns about the quality of inferences based on different measures of instruction.