NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ884314
Record Type: Journal
Publication Date: 2010
Pages: 18
Abstractor: As Provided
Reference Count: 46
ISSN: ISSN-1098-2140
Toward a More Nuanced Approach to Program Effectiveness Assessment: Hierarchical Linear Models in K-12 Program Evaluation
Newton, Xiaoxia A.; Llosa, Lorena
American Journal of Evaluation, v31 n2 p162-179 2010
Most K-12 evaluations are designed to make inferences about how a program implemented at the classroom or school level affects student learning outcomes and such inferences inherently involve hierarchical data structure. One methodological challenge for evaluators is linking program implementation factors typically measured at the classroom or teacher level with student outcomes measured at the individual student level. Hierarchical linear modeling (HLM) is ideal for K-12 program evaluations because it can appropriately handle hierarchical data while allowing for a nuanced conceptualization of evaluation questions. The authors illustrate the advantages of HLM with an example of the evaluation of a technology-based reading program for elementary students. HLM enabled evaluators to have a deeper understanding of the relationship between program implementation and program outcomes by (a) providing a better and more proper estimate of the average program outcome, (b) providing an estimate of variation in student outcomes within classrooms, between classrooms, and between schools, and (c) most importantly, by providing a framework for probing what factors are related to variations in student outcomes. Through illustrating the potentials of HLM, the authors aim to advance and expand evaluative tools for establishing empirical evidence regarding program effectiveness in K-12 settings. The authors also hope to bring balance and additional insight into the current methodological discussions about the relationship between program implementation and program impact, rather than merely demonstrating the average effect of a program or policy intervention. (Contains 3 tables and 2 notes.)
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Evaluative
Education Level: Elementary Education; Elementary Secondary Education; Grade 1; Kindergarten
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: California
Identifiers - Assessments and Surveys: Woodcock Reading Mastery Test
IES Cited: ED560919