NotesFAQContact Us
Collection
Advanced
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1055841
Record Type: Journal
Publication Date: 2015
Pages: 4
Abstractor: ERIC
Reference Count: 5
ISBN: N/A
ISSN: ISSN-1536-6367
Taming Inflation Is Never Easy
Braun, Henry
Measurement: Interdisciplinary Research and Perspectives, v13 n1 p31-34 2015
Many states have implemented, or are implementing, accountability systems for schools and educators that incorporate test-based indicators. There is substantial disagreement on the merits of this trend, but the general consensus appears to be that value-added scores or median (mean) growth percentiles contain useful information with respect to school and/or educator contributions to student progress on achievement tests (Braun, Chudowsky, & Koenig, 2010). Utility, however, depends critically on the validity of the test scores that are the input to the analytic engines generating such indicators. Koretz (this issue) argues that, more often than not, observed increases in test scores are due to a combination of construct-relevant (e.g., greater mastery of the material) and construct-irrelevant (e.g., various strategies for gaming the test) factors. If the latter constitute a nontrivial component of test score increases, and since they do not reflect genuine learning, then we have an instance of "test score inflation" (TSI). Koretz adduces evidence for this phenomenon by comparing trends in aggregate student performance on high-stakes, mandated state tests and low-stakes tests, such as the National Assessment of Educational Progress (NAEP). Such comparisons are surely confounded by differential motivation and effort (Braun, Kirsch, & Yamamoto, 2011), by differences in the alignment of the test frameworks to the state content standards, or by the widely observed general lack of transfer. Clearly, more research is needed to quantify the magnitude of TSI in various settings. Initial results in this direction are provided by Jennings and Bearak (2014). Nonetheless, the divergence in trends is sufficiently large and widespread to support the contention that TSI may be both real and worrisome. What then should be done? This author argues that we must not only reduce the opportunities to produce TSI, but also lessen the motivation to do so. With respect to the latter, there are at least four steps that can be taken. First, design accountability systems that incorporate multiple indicators of different types, with no one indicator having a disproportionate influence. Ensure that the process of combining indicators into a composite rating is sensible and admits some role for local judgment. Second, build a true system of assessments (with the end-of-course test as the capstone) that generates relevant, timely information throughout the school year. Marry that information with support and capacity development. Next, improve communication and outreach so that all stakeholders have a better understanding of the different indicators and how they operate, both singly and in combination. Finally, perhaps most important and certainly most difficult, build trust among stakeholders and across different levels: educators, school administrators, school district central office and state department of education staff, and state officials. In many states, however, distrust and adversarial relationships appear now to predominate. Without improvement on this step, even substantial progress on the others is unlikely to have more than a very modest impact.
Psychology Press. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Opinion Papers; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A