NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: ED504657
Record Type: Non-Journal
Publication Date: 2009-Feb
Pages: 111
Abstractor: ERIC
Effectiveness of Reading and Mathematics Software Products: Findings From Two Student Cohorts. NCEE 2009-4041
Campuzano, Larissa; Dynarski, Mark; Agodini, Roberto; Rall, Kristina
National Center for Education Evaluation and Regional Assistance
In the No Child Left Behind Act (NCLB), Congress called for the U.S. Department of Education (ED) to conduct a rigorous study of the conditions and practices under which educational technology is effective in increasing student academic achievement. A 2007 report presenting study findings for the 2004-2005 school year, indicated that, after one school year, differences in student test scores were not statistically significant between classrooms that were randomly assigned to use software products and those that were randomly assigned not to use products. School and teacher characteristics generally were not related to whether products were effective. The second year of the study examined whether an additional year of teaching experience using the software products increased the estimated effects of software products on student test scores. The evidence for this hypothesis is mixed. For reading, there were no statistically significant differences between the effects that products had on standardized student test scores in the first year and the second year. For sixth grade math, product effects on student test scores were statistically significantly lower (more negative) in the second year than in the first year, and for algebra I, effects on student test scores were statistically significantly higher in the second year than in the first year. The study also tested whether using any of the 10 software products increased student test scores. One product had a positive and statistically significant effect. Nine did not have statistically significant effects on test scores. Five of the insignificant effects were negative and four were positive. Study findings should be interpreted in the context of design and objectives. The study examined a range of reading and math software products in a range of diverse school districts and schools. But it did not study many forms of educational technology and it did not include many types of software products. How much information the findings provide about the effectiveness of products that are not in the study is an open question. Products in the study also were implemented in a specific set of districts and schools, and other districts and schools may have different experiences with the products. The findings should be viewed as one element within a larger set of research studies that have explored the effectiveness of software products. Three appendixes are included: (1) Second-Year Data Collection and Response Rates; (2) Description of Sample for the 10 Products; and (3) Details of Estimation Methods. (Contains 29 footnotes, 4 figures and 24 tables.
National Center for Education Evaluation and Regional Assistance. Available from: ED Pubs. P.O. Box 1398, Jessup, MD 20794-1398. Tel: 877-433-7827; Web site:
Publication Type: Numerical/Quantitative Data; Reports - Descriptive
Education Level: Grade 1; Grade 4; Grade 6
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Institute of Education Sciences (ED), National Center for Education Evaluation and Regional Assistance
Identifiers - Laws, Policies, & Programs: No Child Left Behind Act 2001
Identifiers - Assessments and Surveys: California Achievement Tests; Iowa Tests of Basic Skills; Otis Lennon School Ability Test; Stanford Achievement Tests
IES Funded: Yes
Grant or Contract Numbers: N/A
What Works Clearinghouse Reviewed: Meets Evidence Standards without Reservations