NotesFAQContact Us
Search Tips
ERIC Number: ED547352
Record Type: Non-Journal
Publication Date: 2012
Pages: 162
Abstractor: As Provided
Reference Count: N/A
ISBN: 978-1-2674-4249-9
A Comparison of Value-Added, Ordinary Least Squares Regression, and the California STAR Accountability Indicators
Black, Aime
ProQuest LLC, Ed.D. Dissertation, University of Southern California
Student achievement to reward or sanction schools. These unadjusted accountability indicators do not account for differences in student or school characteristics that contribute to variations in assessment results. Since the "Coleman Report" (1966), a guiding principle in accountability design has been that educational outcomes data should be used only "after" the effects of institutional characteristics have been statistically removed. Such indices are called adjusted indicators, where an adjustment is either statistical or through aggregation. The purpose of this study is to analyze the reliability (internal consistency and test-retest) and validity (discriminant, convergent, and concurrent) of six available accountability indicator systems: (a) API improvement scores, (b) API[subscript SED] scores (SED = socioeconomically disadvantaged), (c) similar schools scores, (d) LA Times value-added scores, (e) academic growth over time (AGT) value-added scores (ELA and math), and (f) adjusted normal curve equivalent (ANCE) scores (ELA and math). Each system has been proposed as adjuncts to the currently operationalized school status (average achievement) scores. The population included all K-5 elementary schools in LAUSD; specifically grades 2 through 5 for the 2009 and 2010 school years. There were approximately 430 schools and their teachers included in this research study. An initial analysis indicated that unadjusted variables of student achievement were highly correlated to each other and notably and significantly linked to poverty status. In particular, the NCLB status model indicator, the Adequate Yearly Progress (AYP), and the PSAA improvement model indicator, the Academic Performance Index (API), which are used as primary indicators in the California accountability systems to hold schools accountable, were substantially correlated to socioeconomic status. Subsequent to the initial analysis, the reliability and validity of the six different adjusted accountability indicator systems were investigated. A first conclusion is that the only justifiable methods for evaluating schools are ANCE scores and similar school scores that have been adjusted for institutional characteristics using ordinary least squares (OLS) regression. Of the two justifiable methods, ANCE scores can be disaggregated into ELA and math components and therefore are preferred. The third phase of this study examined the reliability (internal consistency and test-retest stability) and the validity (discriminant and concurrent) of adjusted grade Level equivalent (AGLE) scores. AGLE scores are grade level ANCE scores in ELA and math that have been adjusted for institutional characteristics using ordinary least squares (OLS) regression. Results from this study suggested that AGLE scores are reliable and valid for use in holding grade-level teams, which are ignored under NCLB and PSAA, accountable for enhanced learning outcomes. Furthermore, AGLE scores can be simply averaged to generate reliable and valid school composite scores in an NCE metric. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page:]
ProQuest LLC. 789 East Eisenhower Parkway, P.O. Box 1346, Ann Arbor, MI 48106. Tel: 800-521-0600; Web site:
Publication Type: Dissertations/Theses - Doctoral Dissertations
Education Level: Elementary Education; Grade 2; Primary Education; Early Childhood Education; Grade 3; Grade 4; Intermediate Grades; Grade 5; Middle Schools
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: California