NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: ED507314
Record Type: Non-Journal
Publication Date: 2009-Nov
Pages: 91
Abstractor: ERIC
Reference Count: 67
A Multisite Cluster Randomized Trial of the Effects of CompassLearning Odyssey[R] Math on the Math Achievement of Selected Grade 4 Students in the Mid-Atlantic Region. Final Report. NCEE 2009-4068
Wijekumar, Kay; Hitchcock, John; Turner, Herb; Lei, PuiWa; Peck, Kyle
National Center for Education Evaluation and Regional Assistance
In an effort to identify instructional methods that might improve mathematics learning at the grade 4 level when used in a variety of educational settings under typical conditions, the REL Mid-Atlantic research team looked for promising, replicable practices that were being used broadly by teachers in U.S. schools, for which research showed promising results but had not been conducted using methodologies that can establish causal relationships. CompassLearning's Odyssey[R] Math product met all of these criteria. Odyssey Math is a computer-based math curriculum developed by CompassLearning, Inc., to improve math learning for K-12 students. The software consists of a web-accessed series of learning activities, assessments, and math tools. These components constitute the basic framework of the software. CompassLearning professional development trainers presented the learning activities, math tools, and assessments as available options to intervention teachers during the summer professional development session. This study was the first randomized controlled trial to assess the impact of Odyssey Math on student achievement. The study had the statistical power needed to detect a 0.20 effect size and was well designed in that comparable groups were created at baseline and maintained through posttesting. Implementation during the school year was documented and shown to be consistent with typical implementation of the Odyssey Math software. The results from the multilevel model with pretest covariates also indicate that Odyssey Math did not yield a statistically significant impact on end-of-year student achievement. This study generated a statistically unbiased estimate of the effect of Odyssey Math on student achievement when implemented in typical school settings with typical teacher and student use. However, the findings apply only to participating schools, teachers, and students because the study used a volunteer sample. Twelve appendices include: (1) Detailed Professional Development Agenda Sessions; (2) Statistical Power Analysis; (3) Probability of Assignment to Study Conditions; (4) Sample Size from Random Assignment to Data Analysis; (5) Teacher Survey, Fall 2007; (6) Observation Protocols; (7) Odyssey Math Sample Screens; (8) Fidelity Observation Comparisons; (9) Model Variance and Intraclass Correlations; (10) Complete Multilevel Model Results for Research Question 1; (11) Comparison of Assumed Population Parameters for Statistical Power (During Planning Phase) with Corresponding Sample Statistics (During Analysis Phase); and (12) Equations for Multilevel Model Analyses. (Contains 3 figures, 22 tables, 10 exhibits, and 23 footnotes.) [This report was prepared for the National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences under contract with Regional Educational Laboratory Mid-Atlantic administered by Pennsylvania State University.]
National Center for Education Evaluation and Regional Assistance. Available from: ED Pubs. P.O. Box 1398, Jessup, MD 20794-1398. Tel: 877-433-7827; Web site:
Publication Type: Numerical/Quantitative Data; Reports - Evaluative
Education Level: Elementary Education; Grade 4
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: National Center for Education Evaluation and Regional Assistance (ED)
IES Funded: Yes
What Works Clearinghouse Reviewed: Meets Evidence Standards without Reservations
IES Cited: ED532453