NotesFAQContact Us
Search Tips
ERIC Number: ED528756
Record Type: Non-Journal
Publication Date: 2011
Pages: 7
Abstractor: ERIC
Reference Count: 3
The Impact of Indiana's System of Diagnostic Assessments on Mathematics Achievement
Konstantopoulos, Spyros; Miller, Shazia; van der Ploeg, Arie; Li, Cheng-Hsien; Traynor, Anne
Society for Research on Educational Effectiveness
The purpose of the study was to design a rigorous experimental study and collect high quality data to determine the effectiveness of the intervention on student achievement. In particular, the authors examined whether diagnostic assessments implemented by schools in Indiana produced rigorous estimates of their effects on student performance on the state's annual Indiana Statewide Testing for Educational Progress-Plus (ISTEP+) measures. Because the data were produced from a well designed randomized experiment they have high internal validity and justify causal inferences about the intervention effects (Shadish, Cook, & Campbell, 2002). In addition, because their sample included numerous schools (58 schools) from different school districts in Indiana their results should have higher external validity than those from convenient and localized samples. To account for the nesting of students within schools the authors employed two-level models to examine the effectiveness of the intervention. The experiment took place in Indiana in 2009-2010 and included K-8 schools that had volunteered to be part of the intervention the Spring of 2009. The authors used data from the first year of the experiment for grades K through 8. They conducted analysis on the initial number of schools that were randomly assigned to conditions (Intention to Treat or ITT) and on the participating or TOT (Treatment on Treated) schools. The outcome was mathematics scores of ISTEP+. The findings overall suggested that across all grades (K to 8) the treatment effect was positive, but it was not as large and not significantly different from zero. As a result, it seems that the type of diagnostic assessment that Indiana implemented in the sample of schools they evaluated did not improve mathematics achievement that much. The same conclusion holds for the results using grades K to 2 and 3 to 8. However, the within grade analysis revealed that in fifth grade mathematics the treatment effect was significant and not trivial. The treatment effect was consistently as large as one-fifth of a standard deviation and indicated an important annual gain in mathematics achievement. All other estimates were not consistently significant and depended on the specification. Hence, it is unclear that the intervention had any systematic effects on student achievement except for fifth grade mathematics. The estimates produced from the TOT analysis indicated significant treatment effects. However, it is likely that the TOT estimates are biased due to selection and therefore they are not as internally valid as those produced by the ITT analysis.
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail:; Web site:
Publication Type: Reports - Research
Education Level: Elementary Education; Middle Schools
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Location: Indiana